• nserver.exe 100% CPU usage

    By Gord Edge 2 decades ago

    Our server randomly crashes when kspam is running. Once kspam is loaded it could take 10 minutes or 2 days to crash. Nserver.exe will all of a sudden be at 100%. When we remove Extmgr_addins=spam from notes.ini everything is fine. We are using version 1.4b. Have tried newer versions but it won't pick up our mail because we replicate with another server. Any input would be greatly appreciated.

    • Re: crash

      By Tom Lyne 2 decades ago

      Hi, could you add ks_debug=1 to the notes.ini file; restart the server and when it crashes, remove kSpam again and post the ks_log.txt file here.



      Also, when you say you "replicate" the mail, what do you mean exactly?



      Thanks,

      tom

      • Re Crash

        By Nico Vis 2 decades ago

        Do you have rules that reload every hour.

        There were a couple of bugs thate made Domino server crash in 1.4b.

        They were fixed in the next releases, try to look at the description in the next ones.

        Let me know.

        Rgds

        Nico

    • ks_log.txt

      By Gord Edge 2 decades ago

      The first crash was at 9:09am the second crash at 9:44am. The replicating is when our mail goes through our head office at another location.







      Number of rules: 1

      Number of rules: 1

      12/01/2007 09:09:11 AM - Form: Memo

      *SUBJECT:Barrel of Fun Vics Lite White Cheddar Popcorn 4oz (Jan 11,07).xls*<br/>
      *FROM:&quot;wg.jwheeler&quot; &lt;xxxxxxxxxxn@wxxxxxxxxxxn.com&gt;*<br/>
      *MAIL_FROM:xxxxxxxxxxn@wxxxxxxxxxxnc.com*<br/>
      *BODY LENGTH: 2166* <br/>
      *ATTACHMENT: barrel of fun vics lite white cheddar popcorn 4oz (jan 11,07).xls*<br/>
      *Scanning message*<br/>
      

      ERROR 5.2.1: kSpam.bload calculating probabilties or not loaded.

      ERROR 5.1.1: Bayesian data not loaded. P:0.0000



      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      12/01/2007 09:39:33 AM - Form: Memo

      *SUBJECT:Confirmation - Lab Request # 07R011263*<br/>
      *FROM:formsdb@xxxxxxxxxxn.com*<br/>
      *MAIL_FROM:www-data@xxxxxxxxxxn.com*<br/>
      *BODY LENGTH: 395* <br/>
      *Scanning message*<br/>
      

      ERROR 5.2.1: kSpam.bload calculating probabilties or not loaded.

      ERROR 5.1.1: Bayesian data not loaded. P:0.0000



      12/01/2007 09:39:33 AM - Form: Memo

      *SUBJECT:**New Lab Request # 07R011263*<br/>
      *FROM:formsdb@xxxxxxxxxxn.com*<br/>
      *MAIL_FROM:www-data@xxxxxxxxxxn.com*<br/>
      *BODY LENGTH: 84* <br/>
      *Scanning message*<br/>
      

      ERROR 5.2.1: kSpam.bload calculating probabilties or not loaded.

      ERROR 5.1.1: Bayesian data not loaded. P:0.0000



      12/01/2007 09:44:38 AM - Form: Memo

      *SUBJECT:Happy Birthday - Fellow Cap!*<br/>
      *FROM:Natalie &lt;scamac@xxxxxxxxxxn.com&gt;*<br/>
      *MAIL_FROM:scamac@xxxxxxxxxxn.com*<br/>
      *BODY LENGTH: 667* <br/>
      *ATTACHMENT: th_thebday2.gif*<br/>
      *Scanning message*<br/>
      

      ERROR 5.2.1: kSpam.bload calculating probabilties or not loaded.

      ERROR 5.1.1: Bayesian data not loaded. P:0.0000



      12/01/2007 09:44:40 AM - Form: Memo

      *SUBJECT:Lessons from Dogs*<br/>
      *FROM:&quot;Nancy xxxxx&quot; &lt;nwilson@xxxxxxxxxxn.com&gt;*<br/>
      *MAIL_FROM:nwilson@xxxxxxxxxxn.com*<br/>
      

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      Reloading configuration…

      freeing rule list

      freed rule list

      loading rule list…

      Number of rules: 1

      12/01/2007 10:15:39 AM - Form: Memo

      *SUBJECT:Renee thought you would like this*<br/>
      *FROM:taf@xxxxxxxxxxn.com*<br/>
      *MAIL_FROM:xxxxxxxxxxn@sympatico.ca*<br/>
      *BODY LENGTH: 8100* <br/>
      *Scanning message*<br/>
      

      ERROR 5.2.1: kSpam.bload calculating probabilties or not loaded.

      ERROR 5.1.1: Bayesian data not loaded. P:0.0000

      • Re:

        By Tom Lyne 2 decades ago

        Could you post your configuration too,



        thanks,

        tom

    • configuration

      By Gord Edge 2 decades ago

      Instance ID

      This is a string of six characters or less, should be the same for every server in your organisation (KS_IID). Ourcompk

      Default action

      This is the default action to be taken when one of the hard coded rules is matched (KS_DEFAULTACTION). 3 - Copy & Deny

      Default probability increase

      The default probability increase, only used if increase probability is selected for the default action (KS_DEFAULT_PROB_INC). 0%

      Mark messages with a reason field?

      Add KS_REASON item to an email if a rule is matched (KS_MARK). Yes

      Block messages that are in Domino blacklist?

      The default action is applied if enabled. To use this feature the blacklist feature must be enabled with "log and tag" on the Domino server.

      (KS_BLACKLIST). Yes

      Mark messages with an allow reason field?

      Add KS_ALLOW_REASON item to an email if an allow rule is matched (KS_MARK_ALLOW). Yes

      Reload configuration every hour?

      (KS_RELOAD). Yes

      Show statistics?

      Log statistics under smtp.kSpam.* (KS_STATS). Yes

      Minimum "From:" header length?

      Minimum length of the From: header (KS_MIN_FROM_LENGTH). 0

      Maximum numbers in sender's username?

      Maximum number of integers in the sender's username (KS_MAX_FROM_INTS). 0

      First character in "From:" header must not be a number?

      (KS_FILTER_FROM_INT). No

      Subject must not be empty?

      (KS_FILTER_SUBJECT). Yes

      Other forms to scan?

      Forms other than Memo and Reply delimited by commas (KS_INTERESTING_FORMS).

      Add recipients list to copied and denied messages?

      Add KS_RECIPIENTS readers field to denied messages, username in email address must me included in the recipients username field in their person document. ( KS_RECIPIENTS). No

      Copied mail database

      Database to copy copied messages to, default is mailspam.nsf (KS_COPIED_DB).

      Turn on debugging?

      Create log in ks_debug.txt (KS_DEBUG). Yes

      • Re: config

        By Tom Lyne 2 decades ago

        Are you using the new config database with the old kSpam version? (Not that it would cause the crash)



        So you don't have the Bayesian feature turned on?



        Do you have anything in mailspam.nsf?



        What verison is your Domino server, and on what OS?



        -tom

        • OS and Notes Version

          By Gord Edge 2 decades ago

          We are running windows 2000 server (service pack4) and version 6.01 of Lotus Domino. We are using all the original Files from version 1.4b of KSPAM.

          • OK

            By Tom Lyne 2 decades ago

            Do you have any "static rules", i.e. wordlists?



            Do any of these rules contain regular expressions by any chance?



            Rgds,

            Tom

          • OK

            By Tom Lyne 2 decades ago

            Do you have any "static rules", i.e. wordlists?



            Do any of these rules contain regular expressions by any chance?



            Rgds,

            Tom

          • And....

            By Tom Lyne 2 decades ago

            Could you post the result of "show config smtp.kspam.*"



            Thanks,

            Tom

          • static rules / config

            By Gord Edge 2 decades ago

            We have no static rules. When I typed the show config smtp.kspam I got the following "Invalid Value for Command option"

            • My mistake

              By Tom Lyne 2 decades ago

              It should have been "show stat smtp.kspam.*"



              Also, you don't have any static rules or Bayesian filtering turned on?



              Can you add one static rule, it doesn't matter what's in it, but I suggest you do something along the lines of:



              subject contains: test

              action: allow



              This won't affect anything as it's a single allow rule.



              -tom

              • show stat smtp.kspam

                By Gord Edge 2 decades ago

                This is the message that came up in the console "SMTP not found in Server statistics table"

                • static rules / bayesian filtering

                  By Gord Edge 2 decades ago

                  Bload currently is not running. Everytime we load it the CPU reaches 100% after a few hours.

                  Here's a deny rule we have



                  Deny Rule:



                  Priority: 100

                  From contains:

                  To contains:

                  Subject contains: spamtest

                  Body contains:

                  Attachments:



                  Action: Copy & Deny

                  Mark with:

                  Probability Increase: 0



                  Here's the rest of our config

                  Bayesian filter enabled

                  Enable the Bayesian filter (KS_BAYESIAN_FILTER). Yes

                  Token reload period

                  Period of time between recalculating probabilities (KS_BL_PERIOD). 360

                  Probability boundary

                  Boundary probability at which email is considered spam (KS_BAYESIAN_BOUNDARY). 90%

                  Mark messages with token list and probability?

                  Add KS_BL_PROB and KS_BL_TOKENS to incoming emails (KS_BAYESIAN_MARK). Yes

                  Good message ratio

                  Ratio of good emails passing through the server before an email is copied to the good mail database (KS_BAYESIAN_RATIO). 10

                  Bayesian action

                  Action to take when an emails are probably spam (KS_BAYESIAN_ACTION). 3 - Copy & Deny

                  Mark with

                  Text to mark emails with is the default Bayesian action is mark. (KS_BAYESIAN_ACTION_MARK_WITH).

                  Tokens to ignore

                  Tokens to ignore when calculating probabilities (KS_BL_IGNORE).

                  Dump token lists to file

                  Write token lists to files goodlist.txt and spamlist.txt (KS_BL_DUMPLISTS). No

                  Preparation setting 1

                  All emails that pass all rules without being matched are placed in the mailgood.nsf database (KS_BAYESIAN_PREP). No

                  Preparation setting 2

                  All emails with a probability greater than 90% are copied to mailspam.nsf, all emails with a probability of less than 10% are copied to mailgood.nsf (KS_BAYESIAN_PREP_2). No

                  Turn on debugging?

                  Create bload.txt log file (KS_BL_DEBUG). Yes

                  • Stats and stuff...

                    By Tom Lyne 2 decades ago

                    Can you try "show stat", does anything appear? Not that it's "show stat smtp.kspam", but you can do "show stat smtp.*".



                    So it's bload which is crashing rather than the addin? I must have missed that in your previous posts.



                    How many documents do you have in mailspam.nsf and mailgood.nsf?



                    Rgds,

                    Tom

                    • stats and stuff

                      By Gord Edge 2 decades ago

                      We have 277 in mailgood and 200 in mailspam. When we load kspam, the server still crashes. When bload is loaded it crashes more frequently.



                      This is what we get when show stat smtp



                      SMTP.Command.DATA = 116

                      SMTP.Command.EHLO = 115

                      SMTP.Command.MAIL = 116

                      SMTP.Command.QUIT = 115

                      SMTP.Command.RCPT = 126

                      SMTP.Command.RSET = 1

                      SMTP.kSpam.Bayesian.Action = Copy & Deny

                      SMTP.kSpam.Bayesian.Boundary = 90

                      SMTP.kSpam.Bayesian.Filter = On

                      SMTP.kSpam.Bayesian.Mark = On

                      SMTP.kSpam.Bayesian.Prep = Off

                      SMTP.kSpam.Bayesian.Prep_2 = Off

                      SMTP.kSpam.Bayesian.Ratio = 10

                      SMTP.kSpam.Conf.AddRecipientsItem = Off

                      SMTP.kSpam.Conf.Debug = On

                      SMTP.kSpam.Conf.DefaultAction = Copy & Deny

                      SMTP.kSpam.Conf.DefaultProbIncrease = 0

                      SMTP.kSpam.Conf.FilterFromInt = Off

                      SMTP.kSpam.Conf.IID = Gak

                      SMTP.kSpam.Conf.Mark = On

                      SMTP.kSpam.Conf.MaxNumbersInFrom = 0

                      SMTP.kSpam.Conf.MinFromLength = 0

                      SMTP.kSpam.Conf.Reload = On

                      SMTP.kSpam.LastMinorError = 16/01/2007 09:48:48 AM: ERROR 5.2.1: kSpam.bload calculating probabilties or not loaded.

                      SMTP.kSpam.Version = 1.40 beta 0

                      SMTP.MessagesProcessed = 116

                      SMTP.MsgRecipients.Ave = 1

                      SMTP.MsgRecipients.Max = 6

                      SMTP.MsgRecipients.Min = 1

                      SMTP.MsgRecipients.Total = 126

                      SMTP.MsgSizeKBytes.Ave = 157

                      SMTP.MsgSizeKBytes.Max = 5554

                      SMTP.MsgSizeKBytes.Min = 1

                      SMTP.MsgSizeKBytes.Total = 18295

                      SMTP.Server.Running = TRUE

                      SMTP.SessionDuration.15Sec_to_20Sec = 1

                      SMTP.SessionDuration.20Sec_to_25Sec = 1

                      SMTP.SessionDuration.5Sec_to_10Sec = 1

                      SMTP.SessionDuration.Max = 23

                      SMTP.SessionDuration.Min = 0

                      SMTP.SessionDuration.Under_5Sec = 112

                      SMTP.Sessions.Inbound.Accept.Queue = 0

                      SMTP.Sessions.Inbound.Active = 0

                      SMTP.Sessions.Inbound.Active.SSL = 0

                      SMTP.Sessions.Inbound.BytesReceived = 18,709,323

                      SMTP.Sessions.Inbound.BytesSent = 57,931

                      SMTP.Sessions.Inbound.Peak = 2

                      SMTP.Sessions.Inbound.Peak.SSL = 0

                      SMTP.Sessions.Inbound.Total = 115

                      SMTP.Sessions.Inbound.Total.SSL = 0

                      SMTP.Sessions.Inbound.Total.SSL.Bad_Handshake = 0

                      SMTP.Sessions.Outbound.Active = 0

                      SMTP.Sessions.Outbound.Active.SSL = 0

                      SMTP.Sessions.Outbound.BytesReceived = 94,179

                      SMTP.Sessions.Outbound.BytesSent = 27,281,655

                      SMTP.Sessions.Outbound.Peak = 5

                      SMTP.Sessions.Outbound.Peak.SSL = 0

                      SMTP.Sessions.Outbound.Total = 227

                      SMTP.Sessions.Outbound.Total.SSL = 0

                      SMTP.Sessions.Threads.Busy = 0

                      SMTP.Sessions.Threads.Idle = 2

                      SMTP.Sessions.Threads.InThreadPool = 2

                      SMTP.Sessions.Threads.Peak = 2

                      • Mmm

                        By Tom Lyne 2 decades ago

                        I think this bug was one fixed in 1.5. It sounds like it's the configuration loading problem.



                        What is the reason you couldn't upgrade?

                        • Upgrade - no go

                          By Gord Edge 2 decades ago

                          We tried version 1.5 and 1.6B. Both of them when loaded weren't picking up any mail at all even with rules set.



                          We disabled Norton Antivirus yesterday, loaded kspam and bload. No problems yet. I'll let you know if it crashes.



                          Thanks again for all your help.

                          • Re: Norton

                            By Tom Lyne 2 decades ago

                            Does Norton run an extension manager addin?



                            Does the SMTP task run on this server?



                            Rgds,

                            Tom

                            • By Gord Edge 2 decades ago

                              Norton does not run an extension manager addin. Yes, the SMTP task does run on the server.



                              So far kspam with bload loaded is still running fine. It's been 2 days now. I think that's the longest it's ever gone for without crashing.

                              • Crashed Again....

                                By Gord Edge 2 decades ago

                                Server just crashed again. Nserver.exe 100% CPU usage. I guess Norton wasn't the cause after all.

                                • extensions

                                  By Tom Lyne 2 decades ago

                                  Do you have any other extensions running?



                                  If kSpam crashes, it's normally nsmtp.exe which would show 100% processor usage, although it could be related to a bug which was fixed in 1.5.



                                  Can you email me at:



                                  local: kspam

                                  domain: tomlyne.com



                                  Maybe I can add something to help you out.



                                  -tom