Anomaly detection based on processes executed within a network

- Exabeam, Inc.

A cybersecurity system, method, and computer program is provided for detecting whether an entity's collection of processes during an interval is abnormal compared to the historical collection of processes observed for the entity during previous intervals of the same length. Logs from a training period are used to calculate global and local risk probabilities for each process based on the process's execution history during the training period. Risk probabilities may be computed using a Bayesian framework. For each entity in a network, an entity risk score is calculated by summing the applicable risk probabilities of the unique processes executed by the entity during an interval. An entity's historical risk scores form a score distribution. If an entity's current score is an outlier on the historical score distribution, an alert of potentially malicious behavior is generated with respect to the entity. Additional post-processing may be performed to reduce false positives.

Skip to: Description  ·  Claims  ·  References Cited  · Patent History  ·  Patent History
Description
RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No. 62/609,121, filed on Dec. 21, 2017, and titled “Anomaly Detection Based on Process Executed within a Network,” the contents of which are incorporated by reference herein as if fully disclosed herein.

BACKGROUND OF THE INVENTION 1. Field of the Invention

This invention relates generally to security analytics in computer networks and, more specifically, to anomaly detection based on processes executed within a network.

2. Description of the Background Art

Most of data breaches from enterprises and government organizations involve attackers leveraging stolen user credentials. Once credentials are stolen, they are used to gain access to assets where attackers may issue commands for actions to achieve what they want. Example actions may include privilege escalation and performing reconnaissance.

Cyber attackers are increasingly using “living off the land” attacks. Such attacks often involve using commands/process that an IT administrator would regularly use. These types of attacks can be more difficult to detect than those that involve installing malicious files on the target computer. Therefore, there is demand for a system that better detects living-off-the-land attacks.

SUMMARY OF THE DISCLOSURE

The present disclosure relates to a cybersecurity system, method, and computer program for detecting whether an entity's collection of processes during a period of time is abnormal compared to the historical collection of processes observed for the entity during past periods of the same length. In certain embodiments, the period of time is a day, and the system determines whether an entity's daily collection of processes is anomalous compared to a historical collection of daily processes observed for the entity.

In one embodiment, the method involves a training phase and a prediction phase, as described below.

Training Phase

The system obtains a certain amount of days of end point logs that track processes executed on end points in the network. The system uses the logs to identify certain processes executed by entities in the network during the training period. An entity may be a user or a host computer.

For each of a plurality of monitored processes, the system calculates a global risk probability of the process being used for a malicious attack, wherein a global risk probability for a process is based on the execution history of the process across all entities during the training period. For each unique monitored process-entity combination observed in the training period, the system calculates a local risk probability of the process being used on the entity for a malicious attack, wherein the local risk probability is based on the global risk probability of the process and the execution history of the process on the entity during the training period. In the preferred embodiment, a Bayes framework is used to calculate the global and local risk probabilities.

The system uses the local risk probabilities for entity/process combinations to compute a plurality of training-period risk scores for each entity. Each training-period risk score for an entity is obtained by summing up the local risk probability of each unique monitored processes executed by the entity during an interval within the training period. For example, for each day in the training period, the system may compute a daily risk score for each entity based on the unique monitored process executed by the entity that day.

Prediction Phase

During a prediction phase, the system computes a prediction-period risk score for each entity by summing up a risk score of each unique monitored process executed by the entity during the prediction period. For each process executed by an entity during the prediction period, the system determines whether the process/entity combination was observed during the training phase. If so, the risk score for the process is the local risk probability for the process/entity combination (e.g., the local Bayes risk for the process/entity combination). Otherwise, the risk score for the process is the process' global risk probability (e.g., the global Bayes risk).

For each entity, the system determines whether the entity's prediction-period risk score is anomalous compared to the entity's training-period risk scores. The system displays alerts in a user interface for anomalous prediction-period risk scores. Additional post-processing may be performed to reduce false positive alerts.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating inputs and outputs to the training phase and prediction phase according to one embodiment.

FIG. 2 is a flowchart illustrating a method performed with respect to a training phase for obtaining a historical collection of risk scores for an entity based on processes executed by the entity.

FIG. 3 is a flowchart illustrating a method performed with respect to a prediction phase for determining whether the processes executed by an entity during a prediction period are anomalous.

FIGS. 4a and 4b are examples of a daily risk score plot and associated histogram for an entity.

 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present disclosure relates to a system, method, and computer program for detecting whether an entity's collection of processes used during a period of time is abnormal compared to a historical collection of processes observed during past periods of the same length. Examples of processes are commands. An entity may be a user or a host computer. The method is performed by a computer system that monitors for potential cyberattacks to networks (“the system”). There are two phases in performing the method: (1) a training period in which the system determines an entity's historical set of daily (or other time period) processes and associated cybersecurity risk scores, and (2) a prediction period in which the system determines, for each entity, whether the cumulative cybersecurity risk associated with an entity's collection of process on a given day (or other period of time) is anomalous compared to the entity's historical distribution of cybersecurity risks scores. Both phases are described below.

1. Training

An entity's historical set of processes during defined intervals is determined with respect to a training period. For example, the training period may be a sliding window of N days (e.g., 30-60 days), and each interval may be a 24-hour day. As another example, the training period may be a sliding window of N weeks, and each interval may be one week.

FIG. 1 illustrates inputs and outputs to the training phase. In one embodiment, the inputs 110 to the training phase are as follows:

    • A certain number of days (e.g., 30-60) of endpoint logs that track executed processes on endpoints (e.g., Carbon Black logs); and
    • A list of processes. In one embodiment, the list includes processes that are regularly used for legitimate purposes, but also have previously been used in malicious attacks. The list of processes is referred to herein as “the monitored processes.”

In one embodiment, the outputs 120 to the training phase are as follows:

    • For each process in the list, a global risk probability (also referred to herein as a “global risk score”);
    • A local risk probability for each process/entity combination observed in training (also referred to herein as a “local risk score”); and
    • A historical set of risk scores for each entity, wherein each risk score corresponds to an interval within the training period. Each of the historical risk scores is obtained by summing the local risk probabilities of process executed by the entity during an interval within the training period (e.g., a daily risk score for each entity for each training day).

FIG. 2 illustrates a method for obtaining the above-listed outputs. The steps are described with respect to sections 1.1-1.3 below.

1.1 Compute Global Risk Probability for Each Monitored Process Over all Entities (Training Phase)

For each of the monitored processes, the system calculates a global risk probability of the process being used for a malicious attack (step 210) (referred to herein as the “global risk probability” or the “global risk score”). The global risk probability is based on the execution history of the process across all entities during the training period. In the preferred embodiment, a Bayesian framework is used to calculate the global risk probability. For example, the following Bayes formula may be used to calculate the global risk probability for a process (process=C below):

p ( M ⁢ C ) = p ( C ⁢ M ) ⁢ p ⁡ ( M ) p ( C ⁢ M ) ⁢ p ⁡ ( M ) + p ( C ⁢ L ) ⁢ p ⁡ ( L ) ,
where
p(M)=prior probability of any event being malicious;
p(L)=prior probability of any event being legitimate;
p(M|C)=probability of any event being malicious given a process is executed. “C” represents process in the equations herein;
p(L|C)=probability of any event being legitimate given a process is executed;
p(C|M)=probability of a process C being in the malicious population; and
p(C|L)=probability of a process C being in the legitimate population.
Assuming p(M)=p(L)=0.5, we have

p ( M ⁢ C ) = p ( C ⁢ M ) p ( C ⁢ M ) + p ( C ⁢ L )

p ( C = c i ⁢ M ) = 1 n .
where n equals to number of unique processes in the malicious population, which is the same as the number of unique process that are monitored. In an alternate embodiment, the probability of a process C being in the malicious population, p(C|M), is manually assigned by a system administrator or domain expert based on past experience.

1.2 Compute Local Bayes Risk for Each Process/Entity Combination Observed in Training (Training Phase)

The system tracks which entities executed the monitored processes during the training phase. For each unique process-entity combination observed, the system calculates a local risk probability of the process being used on the entity for a malicious attack (step 220) (referred to herein as “a local risk probability” or a “local risk score”). The local risk probability for a given process-entity combination is based on the global risk probability for the process and the execution history of the process on the entity during the training period. In the preferred embodiment, a Bayesian framework is used to calculate the local risk probability. For example, the following Bayes formula may be used to calculate the local risk probability for a process (process=C below)/host (host=H) combination:

p ( M ⁢ C , H ) = p ( C ⁢ M , H ) ⁢ p ( M ⁢ H ) p ( C ⁢ M , H ) ⁢ p ⁡ ( M ⁢ H ) + p ( C ⁢ L , H ) ⁢ p ( L ⁢ H )
Where p(M|C,H)=probability of any event being malicious given a process “C” is executed on host “H.”
Assuming p(M|H)=p(M|C), we have

p ( M ⁢ C , H ) = p ( C ⁢ M , H ) ⁢ p ( M ⁢ C ) p ( C ⁢ M , H ) ⁢ p ⁡ ( M ⁢ C ) + p ( C ⁢ L , H ) ⁢ p ( L ⁢ C ) ,
where p(M|C) is the computed global risk and p(L|C)=1−p(M|C)

p ( C = c i ⁢ L , H = h j ) = n i , j ∑ i = 0 n j ⁢ n i , j ,
where ni,j equals to number of unique pairs of (date, hour) when process ci was executed at host hj

p ( C = c i ⁢ M , H = h j ) = 1 n j ,
where nj equals to number of unique commands executed in host hj. In an alternate embodiment, the probability of a process C being in the malicious population on host H, p(C|M, H), is manually assigned by a system administrator or domain expert based on past experience.
In the above formulas, a host computer is the entity, but the above formulas can apply to a user as the entity.

1.2 Compute a Collection of Historical Risk Scores of Each Entity in History by Periodically Summing Up the Local Risk Scores for Each Process Executed by the Entity (Training Phase)

The system computes a historical set of risk scores for each entity based on monitored processes executed by the entity within the training period (step 230). Each historical risk score for an entity is obtained by summing up the local risk probabilities of the unique monitored processes executed by the entity during an interval within the training period. In other words, each entity's risk score is the sum of the local risk scores of the processes executed by the entity during an interval (e.g., a 24-hour day) within the training period. For example, if the interval is a 24-hour day, then for each training day, the system computes a daily risk score for each entity by summing up the local risk score of each unique (and monitored) process executed by the entity that training day. The collection of risk scores for an entity from the training period form a risk score distribution that is used to determine whether the entity's risk score in a future time period (i.e., a prediction period) is anomalous.

2. Prediction

After the training period, the output of the training can be used to identify anomalous process usage by an entity during a prediction period (e.g., a 24-hour day). FIG. 1 illustrates the inputs and outputs to the prediction phase. In one embodiment, the inputs 130 to the prediction phase are as follow:

    • For each process monitored in the training phase, a global risk probability for the process;
    • A local risk probability for each process/entity combination observed in training; and
    • A historical set of risk scores for each entity from the training phase (e.g., a daily risk score for each entity for each training day).
    • Endpoint logs for the prediction phase

In one embodiment, the output 140 of the prediction phase is as follow:

    • Any entities identified as having anomalous process behavior during the prediction period (e.g., a day).

FIG. 3 illustrates a method for obtaining the above-listed output. The steps are described with respect to sections 2.1-2.4 below.

2.1 Compute Risk Score for Each Entity During a Prediction Period

The system computes a risk score for each entity by summing up the risk score of each unique process executed on the entity during a prediction period (e.g., a day) (step 310). For each process executed by an entity during the prediction period, the system determines whether the process/entity combination was observed during the training phase. If so, the risk score for the process is the local risk probability for the process/entity combination (e.g., the local Bayes risk for the process/entity combination). Otherwise, the risk score for the process is the process' global risk probability (e.g., the global Bayes risk).

The prediction period is equal to the length of an interval in the training period (i.e., the interval length with respect to which risk scores are calculated in the training period). For example, if daily risk scores are computed for each entity for each training day, then risk scores are computed on a daily basis in the prediction phase.

2.2 Determine Whether Prediction-Period Risk Score is Anomalous Compared to Risk Score Distribution from Training Phase

The system determines whether any entity's risk score for the prediction period is anomalous compared to the entity's historical risk scores from the training period (step 320). In one embodiment, risk scores are calculated on a daily basis for each entity, and the system computes, for each entity, the p-value of the entity's daily risk score in the prediction phase compared to historical daily risk scores of the entity. If the computed p-value is less than a configured threshold, such as 0.02, the system flags the entity as having anomalous process usage during the prediction period.

For example. FIG. 4a illustrates an example of a plot of a host computer's daily risk scores during a training period. FIG. 4b illustrates a histogram of the host's daily risk scores from the training period. If daily risk score for a prediction day=6.0, then the p-value would be calculated as follows:
p-value=(sum of bin heights with daily score>=6.0)/(sum of all bin heights) in the histogram in FIG. 4b.

2.3 Conduct Post Filtering to Reduce False Positives (Prediction Phase)

In certain embodiments, to reduce false positive alerts, the system conducts post-prediction filtering or ranking on anomalous risk scores for each prediction period (step 330).

There are various ways in which false positive alerts may be reduced. Below are some example ways to rank by determining a degree of anomaly associated with an anomalous prediction period for a user:

1. For each entity that is flagged as having an anomaly during a prediction period because of certain new processes executed by the entity during the prediction period, the system identifies the set of new processes from the prediction period that were not seen in the entity's training history. For example, if a histogram is used to track the processes executed by an entity and the prediction period is a day, the system would identify the processes executed by the entity on the anomalous day that were not previously seen in the entity's histogram.

2. The set of new processes executed by the entity during the prediction period is used to rank the entity's anomaly relative to other entities' anomalies using one of the following:

    • a. The system identifies the number of other entities whose historical models (e.g., histograms) containing this set of processes (i.e., the processes that are new to the subject entity), either all of them or some percentage of the processes. The system then ranks the anomaly for the current prediction period based on this number, where the more entities in the network having executed this set of processes in the past, the less anomalous the set is for the subject entity (i.e., the lower the subject entity's anomalous rank on the subject prediction day); OR
    • b. Rank the anomaly by the ratio: (sum of scores of new processes/current prediction period score); OR
    • c. Rank the anomaly by the ratio: (number of new processes/numbers of current prediction period processes).

3. The system uses the ranking to rank or filter alerts (e.g., select only the x % highest ranked alerts) or to indicate to an administrator a risk level associated with alerts. In certain embodiments, the system may use the ranking to determine whether a risk score rule is triggered or to weight the risk score accordingly.

In an alternate embodiment, the system ranks anomalies by the following ratio: (the current day score−historically closest score)/historically closest score.

2.3 Retraining with Sliding Window

In order to only use the latest data with respect to a prediction period, the system may use a sliding window of N days prior to the prediction period to re-calculate global and local risk scores. For example, the system may use a sliding window of the 60 days prior to the prediction day to recalculate the global and local risk scores.

2.4 Display Alerts for Anomalous Risk Scores

In certain embodiments, the system displays alerts for the x % highest ranking anomalous risk scores in a user interface that displays information related to potential cyber security threats (step 340). For example, the system may display an alert that an entity has anomalous process activity on a current day.

As described below, an entity's prediction-period risk score may also be used to determine if a rule, related to anomalous process execution, triggered in a UEBA system that calculates an overall risk score for a user session.

3. General

The methods described herein are embodied in software and performed by a computer system (comprising one or more computing devices) executing the software. A person skilled in the art would understand that a computer system has one or more memory units, disks, or other physical, computer-readable storage media for storing software instructions, as well as one or more processors for executing the software instructions.

In one embodiment, the computer system is a user/entity behavior analytics (UEBA) system used for network security. An example of a UEBA system is described in the U.S. Pat. No. 9,798,883, titled “System, Method, and Computer Program Product for Detecting and Assessing Security Risks in a Network, the contents of which are incorporated by reference herein. The output of the above-described methods (e.g., whether an entity's daily risk score is anomalous) may be applied to a rules engine of a UEBA system that calculates an overall risk score for an entity or user session. The output may be used to determine whether a rule is triggered for a user session, wherein the triggering of the rule results in an increased risk score for the user session. For example, the methods described herein may be performed by a modeling and anomaly detection module (such as module 1540 in U.S. Pat. No. 9,798,883, incorporated herein above) whose output is received by a rules engine that determines whether risk rules are triggered.

As will be understood by those familiar with the art, the invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the above disclosure is intended to be illustrative, but not limiting, of the scope of the invention.

Claims

1. A method, performed by a computer system, for detecting whether an entity's collection of processes used during a prediction period is abnormal compared to a historical set of processes observed, the method comprising:

performing the following with respect to a training period: for each of a plurality of monitored processes, calculating a global risk probability of the process being used for a malicious attack, wherein a global risk probability for a process is based on the execution history of the process across all entities during the training period; for each unique monitored process-entity combination observed in the training period, calculating a local risk probability of the process being used on the entity for a malicious attack, wherein the local risk probability is based on the global risk probability of the process and the execution history of the process on the entity during the training period; computing a plurality of training-period risk scores for each entity, wherein each training-period risk score for an entity is obtained by summing up the local risk probabilities of the monitored processes executed by the entity during an interval within the training period;
performing the following during the prediction period: computing a prediction-period risk score for each entity by summing up the risk score of each unique monitored process executed by the entity during the prediction period, wherein for each monitored process-entity combination observed during the prediction period the risk score is the local risk probability for the monitored process-entity combination in response to the monitored process-entity combination being observed during the training period, and the risk score is the global risk probability for the monitored process-entity combination in response to the combination not being observed during the training period; for each entity, determining whether the entity's prediction-period risk score is anomalous compared to the entity's training-period risk scores; and displaying alerts in a user interface for anomalous prediction-period risk scores.

2. The method of claim 1, wherein a Bayesian framework is used to calculate the global and local risk probabilities.

3. The method of claim 1, wherein determining whether an entity's prediction-period risk score is anomalous comprises:

calculating a p-value of the entity's prediction-period risk score compared to the training-period risk scores for the entity;
comparing the p-value to a threshold; and
determining the entity's prediction-period risk score is anomalous in response to the p-value being below the threshold.

4. The method of claim 1, further comprising ranking anomalous risk scores for the prediction period according to a degree of anomaly.

5. The method of claim 4, wherein alerts for anomalous risk scores include a risk level for each score based on the score's ranking.

6. The method of claim 4, wherein alerts are displayed only for a subset of the anomalous risk scores, and the subset is selected based on the rankings.

7. The method of claim 1, wherein the prediction period is a day and wherein the training-period risk scores are daily risk scores for each day in the training period.

8. The method of claim 1, wherein the training period is a sliding window of N days, and wherein the prediction period is a sliding window of one day after each training period.

9. A non-transitory computer-readable medium comprising a computer program, that, when executed by a computer system enables the computer system to perform the following method for detecting whether an entity's collection of processes used during a prediction period is abnormal compared to a historical set of processes observed, the method comprising:

performing the following with respect to a training period: for each of a plurality of monitored processes, calculating a global risk probability of the process being used for a malicious attack, wherein a global risk probability for a process is based on the execution history of the process across all entities during the training period; for each unique monitored process-entity combination observed in the training period, calculating a local risk probability of the process being used on the entity for a malicious attack, wherein the local risk probability is based on the global risk probability of the process and the execution history of the process on the entity during the training period; computing a plurality of training-period risk scores for each entity, wherein each training-period risk score for an entity is obtained by summing up the local risk probabilities of the monitored processes executed by the entity during an interval within the training period;
performing the following during the prediction period: computing a prediction-period risk score for each entity by summing up the risk score of each unique monitored process executed by the entity during the prediction period, wherein for each monitored process-entity combination observed during the prediction period the risk score is the local risk score for the monitored process-entity combination in response to the monitored process-entity combination being observed during the training period, and the risk score is the global risk score for the monitored process-entity combination in response to the combination not being observed during the training period; for each entity, determining whether the entity's prediction-period risk score is anomalous compared to the entity's training-period risk scores; and displaying alerts in a user interface for anomalous prediction-period risk scores.

10. The non-transitory computer-readable medium of claim 9, wherein a Bayesian framework is used to calculate the global and local risk probabilities.

11. The non-transitory computer-readable medium of claim 9, wherein determining whether an entity's prediction-period risk score is anomalous comprises:

calculating a p-value of the entity's prediction-period risk score compared to the training-period risk scores for the entity;
comparing the p-value to a threshold; and
determining the entity's prediction-period risk score is anomalous in response to the p-value being below the threshold.

12. The non-transitory computer-readable medium of claim 9, further comprising ranking anomalous risk scores for the prediction period according to a degree of anomaly.

13. The non-transitory computer-readable medium of claim 12, wherein alerts for anomalous risk scores include a risk level for each score based on the score's ranking.

14. The non-transitory computer-readable medium of claim 12, wherein alerts are displayed only for a subset of the anomalous risk scores, and the subset is selected based on the rankings.

15. The non-transitory computer-readable medium of claim 9, wherein the prediction period is a day and wherein the training-period risk scores are daily risk scores for each day in the training period.

16. The non-transitory computer-readable medium of claim 9, wherein the training period is a sliding window of N days, and wherein the prediction period is a sliding window of one day after each training period.

17. A computer system for detecting whether an entity's collection of processes used during a prediction period is abnormal compared to a historical set of processes observed, the system comprising:

one or more processors;
one or more memory units coupled to the one or more processors, wherein the one or more memory units store instructions that, when executed by the one or more processors, cause the system to perform the operations of:
performing the following with respect to a training period: for each of a plurality of monitored processes, calculating a global risk probability of the process being used for a malicious attack, wherein a global risk probability for a process is based on the execution history of the process across all entities during the training period; for each unique monitored process-entity combination observed in the training period, calculating a local risk probability of the process being used on the entity for a malicious attack, wherein the local risk probability is based on the global risk probability of the process and the execution history of the process on the entity during the training period; computing a plurality of training-period risk scores for each entity, wherein each training-period risk score for an entity is obtained by summing up the local risk probabilities of the monitored processes executed by the entity during an interval within the training period;
performing the following during the prediction period: computing a prediction-period risk score for each entity by summing up the risk score of each unique monitored process executed by the entity during the prediction period, wherein for each monitored process-entity combination observed during the prediction period the risk score is the local risk score for the monitored process-entity combination in response to the monitored process-entity combination being observed during the training period, and the risk score is the global risk score for the monitored process-entity combination in response to the combination not being observed during the training period; for each entity, determining whether the entity's prediction-period risk score is anomalous compared to the entity's training-period risk scores; and displaying alerts in a user interface for anomalous prediction-period risk scores.

18. The system of claim 17, wherein a Bayesian framework is used to calculate the global and local risk probabilities.

19. The system of claim 17, wherein determining whether an entity's prediction-period risk score is anomalous comprises:

calculating a p-value of the entity's prediction-period risk score compared to the training-period risk scores for the entity;
comparing the p-value to a threshold; and
determining the entity's prediction-period risk score is anomalous in response to the p-value being below the threshold.

20. The system of claim 17, further comprising ranking anomalous risk scores for the prediction period according to a degree of anomaly.

Referenced Cited
U.S. Patent Documents
5941947 August 24, 1999 Brown et al.
6223985 May 1, 2001 DeLude
6594481 July 15, 2003 Johnson et al.
7181768 February 20, 2007 Ghosh et al.
7624277 November 24, 2009 Simard et al.
7668776 February 23, 2010 Ahles
8326788 December 4, 2012 Allen et al.
8443443 May 14, 2013 Nordstrom et al.
8479302 July 2, 2013 Lin
8484230 July 9, 2013 Harnett et al.
8539088 September 17, 2013 Zheng
8583781 November 12, 2013 Raleigh
8606913 December 10, 2013 Lin
8676273 March 18, 2014 Fujisake
8850570 September 30, 2014 Ramzan
8881289 November 4, 2014 Basavapatna et al.
9055093 June 9, 2015 Borders
9081958 July 14, 2015 Ramzan et al.
9185095 November 10, 2015 Moritz et al.
9189623 November 17, 2015 Lin et al.
9202052 December 1, 2015 Fang
9680938 June 13, 2017 Gil et al.
9690938 June 27, 2017 Saxe et al.
9692765 June 27, 2017 Choi et al.
9760240 September 12, 2017 Maheshwari et al.
9779253 October 3, 2017 Mahaffey et al.
9798883 October 24, 2017 Gil et al.
9843596 December 12, 2017 Averbuch et al.
9898604 February 20, 2018 Fang et al.
10063582 August 28, 2018 Feng et al.
10095871 October 9, 2018 Gil et al.
10178108 January 8, 2019 Lin et al.
10354015 July 16, 2019 Kalchbrenner et al.
10419470 September 17, 2019 Segev et al.
10445311 October 15, 2019 Saurabh et al.
10467631 November 5, 2019 Dhurandhar et al.
10474828 November 12, 2019 Gil et al.
10496815 December 3, 2019 Steiman et al.
10621343 April 14, 2020 Maciejak et al.
10645109 May 5, 2020 Lin et al.
10685293 June 16, 2020 Heimann et al.
10803183 October 13, 2020 Gil et al.
10819724 October 27, 2020 Amiri et al.
10841338 November 17, 2020 Lin et al.
10887325 January 5, 2021 Lin et al.
10944777 March 9, 2021 Lin et al.
11080591 August 3, 2021 van den Oord et al.
11140167 October 5, 2021 Lin et al.
11245716 February 8, 2022 Roelofs
20020107926 August 8, 2002 Lee
20030147512 August 7, 2003 Abburi
20040073569 April 15, 2004 Knott et al.
20060090198 April 27, 2006 Aaron
20070156771 July 5, 2007 Hurley et al.
20070282778 December 6, 2007 Chan et al.
20080028467 January 31, 2008 Kommareddy
20080040802 February 14, 2008 Pierson et al.
20080170690 July 17, 2008 Tysowski
20080262990 October 23, 2008 Kapoor et al.
20080301780 December 4, 2008 Ellison et al.
20090144095 June 4, 2009 Shahi et al.
20090171752 July 2, 2009 Galvin et al.
20090293121 November 26, 2009 Bigus et al.
20100125911 May 20, 2010 Bhaskaran
20100269175 October 21, 2010 Stolfo et al.
20100284282 November 11, 2010 Golic
20110167495 July 7, 2011 Antonakakis et al.
20120278021 November 1, 2012 Lin et al.
20120316835 December 13, 2012 Maeda et al.
20120316981 December 13, 2012 Hoover et al.
20130080631 March 28, 2013 Lin
20130117554 May 9, 2013 Ylonen
20130197998 August 1, 2013 Buhrmann et al.
20130227643 August 29, 2013 Mccoog et al.
20130305357 November 14, 2013 Ayyagari et al.
20130340028 December 19, 2013 Rajagopal et al.
20140007238 January 2, 2014 Magee
20140090058 March 27, 2014 Ward et al.
20140101759 April 10, 2014 Antonakakis et al.
20140315519 October 23, 2014 Nielsen
20150026027 January 22, 2015 Priess
20150039543 February 5, 2015 Athmanathan et al.
20150046969 February 12, 2015 Abuelsaad et al.
20150121503 April 30, 2015 Xiong
20150205944 July 23, 2015 Turgeman
20150215325 July 30, 2015 Ogawa
20150339477 November 26, 2015 Abrams et al.
20150341379 November 26, 2015 Lefebvre et al.
20150363691 December 17, 2015 Gocek et al.
20160005044 January 7, 2016 Moss et al.
20160021117 January 21, 2016 Harmon et al.
20160063397 March 3, 2016 Ylipaavalniemi et al.
20160292592 October 6, 2016 Patthak et al.
20160306965 October 20, 2016 Iyer et al.
20160364427 December 15, 2016 Wedgeworth, III
20170019506 January 19, 2017 Lee et al.
20170024135 January 26, 2017 Christodorescu et al.
20170127016 May 4, 2017 Yu et al.
20170155652 June 1, 2017 Most et al.
20170161451 June 8, 2017 Weinstein et al.
20170213025 July 27, 2017 Srivastav et al.
20170236081 August 17, 2017 Grady Smith et al.
20170264679 September 14, 2017 Chen et al.
20170318034 November 2, 2017 Holland et al.
20170323636 November 9, 2017 Xiao et al.
20180004961 January 4, 2018 Gil et al.
20180048530 February 15, 2018 Nikitaki et al.
20180063168 March 1, 2018 Sofka
20180069893 March 8, 2018 Amit et al.
20180075343 March 15, 2018 van den Oord et al.
20180089304 March 29, 2018 Vizer et al.
20180097822 April 5, 2018 Huang et al.
20180144139 May 24, 2018 Cheng et al.
20180157963 June 7, 2018 Salti et al.
20180165554 June 14, 2018 Zhang et al.
20180190280 July 5, 2018 Cui et al.
20180234443 August 16, 2018 Wolkov et al.
20180248895 August 30, 2018 Watson et al.
20180285340 October 4, 2018 Murphy et al.
20180288063 October 4, 2018 Koottayi et al.
20180288086 October 4, 2018 Amiri et al.
20180307994 October 25, 2018 Cheng et al.
20180322368 November 8, 2018 Zhang et al.
20190014149 January 10, 2019 Cleveland et al.
20190028496 January 24, 2019 Fenoglio et al.
20190034641 January 31, 2019 Gil et al.
20190066185 February 28, 2019 More et al.
20190080225 March 14, 2019 Agarwal
20190089721 March 21, 2019 Pereira et al.
20190103091 April 4, 2019 Chen
20190114419 April 18, 2019 Chistyakov et al.
20190124045 April 25, 2019 Zong et al.
20190132629 May 2, 2019 Kendrick
20190149565 May 16, 2019 Hagi et al.
20190182280 June 13, 2019 La Marca et al.
20190334784 October 31, 2019 Kvernvik et al.
20200021607 January 16, 2020 Muddu et al.
20200082098 March 12, 2020 Gil et al.
20200228557 July 16, 2020 Lin et al.
20200302118 September 24, 2020 Cheng et al.
20210089884 March 25, 2021 Macready et al.
20210125050 April 29, 2021 Wang
20210182612 June 17, 2021 Zeng et al.
20220006814 January 6, 2022 Lin et al.
Other references
  • Bahnsen, Alejandro Correa “Classifying Phishing URLs Using Recurrent Neural Networks”, IEEE 2017.
  • Chen, Jinghui, et al., “Outlier Detection with Autoencoder Ensembles”, Proceedings of the 2017 SIAM International Conference on Data Mining, pp. 90-98.
  • Cooley, R., et al., “Web Mining: Information and Pattern Discovery on the World Wide Web”, Proceedings Ninth IEEE International Conference on Tools with Artificial Intelligence, Nov. 3-8, 1997, pp. 558-567.
  • Goh, Jonathan et al., “Anomaly Detection in Cyber Physical Systems using Recurrent Neural Networks”, IEEE 2017.
  • Kim, Jihyun et al., “Long Short Term Memory Recurrent Neural Network Classifier for Intrusion Detection”, IEEE 2016.
  • Poh, Norman, et al., “EER of Fixed and Trainable Fusion Classifiers: A Theoretical Study with Application to Biometric Authentication Tasks”, Multiple Classifier Systems, MCS 2005, Lecture Notes in Computer Science, vol. 3541, pp. 1-11.
  • Taylor, Adrian et al., “Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks”, IEEE 2016.
  • Taylor, Adrian “Anomaly-Based Detection of Malicious Activity in In-Vehicle Networks”, Ph.D. Thesis, University of Ottawa 2017.
  • Wang, Shuhao et al., “Session-Based Fraud Detection in Online E-Commerce Transactions Using Recurrent Neural Networks”, 2017.
  • Zhang, Ke et al., “Automated IT System Failue Prediction: A Deep Learning Approach”, IEEE 2016.
  • Ioannidis, Yannis, “The History of Histograms (abridged)”, Proceedings of the 29th VLDB Conference (2003), pp. 1-12.
  • DatumBox Blog, “Machine Learning Tutorial: The Naive Bayes Text Classifier”, DatumBox Machine Learning Blog and Software Development News, Jan. 2014, pp. 1-11.
  • Freeman, David, et al., “Who are you? A Statistical Approach to Measuring User Authenticity”, NDSS, Feb. 2016, pp. 1-15.
  • Malik, Hassan, et al., “Automatic Training Data Cleaning for Text Classification”, 11th IEEE International Conference on Data Mining Workshops, 2011, pp. 442-449.
  • Wang, Alex Hai, “Don't Follow Me Spam Detection in Twitter”, International Conference on Security and Cryptography, 2010, pp. 1-10.
Patent History
Patent number: 11423143
Type: Grant
Filed: Dec 20, 2018
Date of Patent: Aug 23, 2022
Assignee: Exabeam, Inc. (Foster City, CA)
Inventors: Derek Lin (San Mateo, CA), Barry Steiman (San Ramon, CA), Domingo Mihovilovic (Menlo Park, CA), Sylvain Gil (San Francisco, CA)
Primary Examiner: Josnel Jeudy
Application Number: 16/228,071
Classifications
Current U.S. Class: Intrusion Detection (726/23)
International Classification: G06F 21/00 (20130101); G06F 21/55 (20130101); G06F 21/56 (20130101); G06F 11/34 (20060101); G06N 7/00 (20060101);