Discovering graymail through real-time analysis of incoming email

Techniques for identifying and processing graymail are disclosed. An electronic message store is accessed. A determination is made that a first message included in the electronic message store represents graymail, including by accessing a profile associated with an addressee of the first message. A remedial action is taken in response to determining that the first message represents graymail.

Skip to: Description  ·  Claims  ·  References Cited  · Patent History  ·  Patent History
Description
CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 17/509,772 entitled DISCOVERING GRAYMAIL THROUGH REAL-TIME ANALYSIS OF INCOMING EMAIL filed Oct. 25, 2021, which claims priority to U.S. Provisional Patent Application No. 63/105,020 entitled DISCOVERING GRAYMAIL THROUGH REAL-TIME ANALYSIS OF INCOMING EMAIL filed Oct. 23, 2020, each of which is incorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

Employees of enterprise organizations receive a variety of types of electronic messages. Some of these messages may be wanted (e.g., legitimate communications made among employees of a given enterprise, or made between employees and entities outside of the enterprise). Others of these messages may be malicious (e.g., attempting to compromise computing infrastructure or defraud the recipient) or otherwise unwanted. Unfortunately, differentiating between various types of messages can be a daunting task, particularly as the number of electronic messages an individual receives on a given day increases. Accordingly, there is an ongoing need for improvements to techniques for managing electronic messages.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.

FIG. 1 illustrates how a conventional filtering service can examine incoming emails to determine which emails, if any, should be prevented from reaching their intended destination.

FIG. 2 depicts an example of a computer-implemented platform (also referred to as a “threat detection platform”) that is designed to identify email messages that constitute graymail and handle those email messages accordingly.

FIG. 3 illustrates an example of how a remediation module can implement a graymail remediation service.

FIG. 4 illustrates an example of how a graymail remediation service can be implemented in a scalable manner.

FIG. 5 is a block diagram illustrating an example of a processing system in which at least some operations described herein can be implemented.

FIG. 6 illustrates an example of a process for managing potential graymail of an organization.

 DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

I. Introduction

The term “graymail” refers to solicited bulk email messages that do not fit the conventional definition of spam, typically because the recipient “opted into” receiving those bulk email messages. Recipient interest in this type of mailing tends to diminish, however, so the likelihood that recipients will report graymail as spam often increases over time. In comparison to spam, graymail can be identified by one or more of the following traits (though need not have all three of the following traits):

    • Solicited—The recipient requested to receive the email message by opting in, either directly or indirectly, via a non-obvious mechanism, such as by supplying an email address when registering for an account or registering for a conference.
    • Legitimate—Though the vast majority of graymail may be representative of notices, coupons, social updates, and the like, these email messages are sent by reputable sources who value their relationship with the recipient. Most of these email messages contain an unsubscribe option, with the sender honoring such requests.
    • Content Value—Graymail generally contains content that is targeted to the recipient or the recipient's interests, as opposed to spam that is an indiscriminate bulk distribution of unwanted solicitations, offers, and information.

In addition to the above traits, there is often a timeliness component to graymail. That is, the utility of an email message determined to be representative of graymail will normally expire or become less useful after a period of time. Notifications of sales or upcoming events are examples of graymail with timeliness components, since these email messages are only valid for a period of time even though recipients may (and often do) read these email messages after the period of time has expired.

While all recipients of graymail “opted in,” either knowingly or unknowingly, to receiving bulk email messages, these email messages have varying value to different recipients. Example categories of graymail include (1) promotional messages (e.g., sale notifications, new product notifications, etc., typically targeted to a topic/product the recipient previously indicated an interest in), (2) newsletters, (3) event invitations (e.g., to conferences, fireside chats, etc.), and (4) cold calls from potential vendors/service providers, typically directed to those with purchasing authority (e.g., generated based on the sender reviewing the recipient's website profile or a profile on LinkedIn). In enterprises (also referred to as “businesses” or “organizations”), graymail can be difficult to manage as some recipients will want to receive at least some of these email messages while other recipients will perceive these email messages as spam. For example, a recipient in the purchasing department may want to receive advertisements for product pricing while a recipient in the marketing department may view these advertisements as spam. Similarly, a recipient in the engineering department may want to receive invitations to conferences/speaking opportunities, while a person in the human resources department may view such types of messages as spam. And, while cold calls may appear to be spam to certain employees, employees engaged in external business relations may consider such messages to be of high value (e.g., in establishing new partnerships). Further, different graymail folders can have differently configurable default expirations, e.g., to reflect the time sensitivity such messages typically have. For example, any graymail moved into a “newsletter” folder of a user can have a default expiration of 90 days, while any graymail moved into a “promotions” folder of the user can have a different default expiration if desired (e.g., 30 days).

Described herein are various computer programs and associated computer-implemented techniques for discovering graymail in the incoming email messages received by employees of an enterprise. For the purpose of illustration, assume that an enterprise receives an incoming email message (“email”) addressed to a recipient, such as an employee of the enterprise (or one or more accounts shared by multiple such employees, etc.). To establish the risk posed by the incoming email, its content or context can be analyzed by a scoring module. For example, the scoring module can examine the incoming email to identify the alleged identity of the sender, email address of the sender, content of the subject line, content of the body, attachments, etc. Further information on example ways incoming emails can be examined can be found in U.S. Pat. No. 10,911,489, which is incorporated by reference herein in its entirety. Generally, if the incoming email is determined to represent a risk to the security of the enterprise, then the incoming email is quarantined and not permitted to reach the inbox of the recipient. However, if the scoring module determines that the incoming email does not represent a risk, then the scoring module can determine what actions, if any, are appropriate for dealing with the incoming email.

As part of this risk determination process, the scoring module may determine the likelihood that the incoming email is representative of graymail. If the scoring module determines that the incoming email is not representative of graymail, then the incoming email can be permitted to proceed to the inbox of the recipient. However, if the incoming email is determined to represent graymail, then the scoring module may indicate as much to a remediation module that is responsible for handling the incoming email. As further discussed below, the remediation module can be responsible for implementing a graymail remediation service (“remediation service”) for automatically moving (e.g., using an application programming interface provided by the enterprise's mail system) incoming emails (that are representative of graymail) to appropriate folders, e.g., using native folder infrastructure of a user's mail account (whether those folders are created automatically on behalf of the user, or at the user's direction). A benefit of this approach is that, irrespective of which mail client the user chooses to access their electronic mail, graymail will be automatically sorted into an appropriate folder, without, for example, requiring modification to the mail client. Thus, a user can choose to view messages using a native mail client (e.g., provided by a phone or computer manufacturer, or an operating system provider) or third party mail client (e.g., software provided by an entity other than a device or operating system manufacturer) and receive inbox decluttering benefits described herein.

Referring again to the scenario described above, assume that the scoring module determines that the incoming email is representative of graymail but poses no risk to the security of the enterprise. In this situation, the scoring module can transmit a request to the remediation module to handle the incoming email. Initially, the remediation module may confirm that a graymail folder (also referred to herein as a “promotions folder”) has been created for the recipient. Generally, the graymail folder is accessible through the employee's mailbox similar to other folders, such as sent folders, spam folders, etc. If a graymail folder was previously created, then the remediation module can transfer the incoming email into the graymail folder. However, if a graymail folder was not previously created, then the remediation module can create a graymail folder into which the incoming email is transferred. If the user subsequently moves a message in the promotions folder back to the user's inbox (e.g., as reported by a cloud-based email suite to a threat detection platform), in some embodiments, the remediation module automatically generates a rule (e.g., based on sender domain, address, etc.) to prevent future messages sent to that user by that sender from being re-routed to the user's promotions folder by the remediation module. Similarly, if the user subsequently moves a message in the promotions folder to the user's spam folder, the remediation module can automatically generate a rule to route future messages from that sender to the user's spam folder instead of to the user's promotions folder (or inbox).

Note that more than one graymail folder can be created for a given recipient. For example, the remediation module can create separate folders for different types of graymail (or as subfolders under a more general graymail folder). Thus, a single recipient may have different graymail folders for promotions, newsletters, invitations/speaking engagements, cold calls, etc. The remediation module can determine the appropriate graymail folder based on an output produced by the scoring module. For example, the remediation module may determine the appropriate graymail folder based on whether the scoring module has labeled an incoming email as representative of a promotion, advertisement, or newsletter based on analysis of its content. The scoring module can use a set of heuristics/rules and/or machine learning models to identify graymail. In an example implementation, different types of graymail can be used as ground truth training data to develop a set of models that can collectively identify graymail and further classify the graymail into one of a variety of subcategories. For example, a set of invitations to conferences/other speaking opportunities/etc. can be used to train a graymail subcategory related to such events. As another example, a set of newsletters can be used to train a graymail subcategory related to such content. The set of models (e.g., including ones specific to specific types of graymail and/or ones trained generally on graymail) can be used to classify incoming messages as graymail (and, as applicable, subcategories of graymail) and ultimately place such messages in an appropriate folder or subfolder.

Embodiments may be described herein with reference to certain types of graymail or certain features of incoming email. However, features of those embodiments may be similarly applicable to other types of graymail and other features of incoming email. As an example, while embodiments may be described in the context of a scoring module that determines risk based on the email address of the sender, the scoring module could consider other feature(s) of the incoming email instead of, or in addition to, the email address of the sender.

While embodiments may be described in the context of computer-executable instructions, aspects of the technologies described herein can be implemented via hardware, firmware, or software. As an example, the scoring module and remediation module may be embodied as instruction sets executable by a computer program that offers support for discovering, classifying, and then remediating security threats.

A. Terminology

References in this description to “an embodiment” or “one embodiment” means that the feature, function, structure, or characteristic being described is included in at least one embodiment of the technology. Occurrences of such phrases do not necessarily refer to the same embodiment, nor are they necessarily referring to alternative embodiments that are mutually exclusive of one another.

Unless the context clearly requires otherwise, the terms “comprise,” “comprising,” and “comprised of” are to be construed in an inclusive sense rather than an exclusive or exhaustive sense (i.e., in the sense of “including but not limited to”). The term “based on” is also to be construed in an inclusive sense rather than an exclusive or exhaustive sense. Thus, unless otherwise noted, the term “based on” is intended to mean “based at least in part on.”

The terms “connected,” “coupled,” or any variant thereof is intended to include any connection or coupling between two or more elements, either direct or indirect. The connection/coupling can be physical, logical, or a combination thereof. For example, objects may be electrically or communicatively coupled to one another despite not sharing a physical connection.

The term “module” refers broadly to software components, firmware components, or hardware components. Modules are typically functional components that generate output(s) based on specified input(s). A computer program may include one or more modules. Thus, a computer program may include multiple modules responsible for completing different tasks or a single module responsible for completing all tasks.

When used in reference to a list of multiple items, the term “or” is intended to cover all of the following interpretations: any of the items in the list, all of the items in the list, and any combination of items in the list.

The sequences of steps performed in any of the processes described here are exemplary. However, unless contrary to physical possibility, the steps may be performed in various sequences and combinations. For example, steps could be added to, or removed from, the processes described here. Similarly, steps could be replaced or reordered. Thus, descriptions of any processes are intended to be open-ended.

B. Conventional Filtering Services

Basic filtering services are offered by most email platforms. FIG. 1 illustrates an example of how a conventional filtering service can examine incoming emails to determine which emails, if any, should be prevented from reaching their intended destination. In some instances, an enterprise applies an anti-spam filter 104 to incoming emails received via the Internet, while in other instances another entity, such as an email service, applies the anti-spam filter 104 to the incoming emails on behalf of the enterprise. The term “external emails” may be used to refer to emails received via the Internet 102, while the term “internal emails” may be used to refer to emails that are sent within the enterprise. An example of an internal email is an intra-enterprise email (e.g., an email from one employee to another employee) that is delivered directly to the mailbox of the recipient rather than routed through the mail exchanger (MX) record, external gateways, etc.

Generally, the anti-spam filter 104 is designed to quarantine malicious emails using blacklists of senders, sender email addresses, and Uniform Resource Locators (URLs) that have been detected in past unsolicited emails or defined in policy frameworks created by the enterprise. The term “anti-spam filter,” as used herein, can refer to any legacy email security mechanism capable of filtering incoming emails, including secure email gateways (SEGs) (also referred to as “gateways”). For example, the enterprise (or the email service) can maintain a list of sender email addresses from which malicious email has been received in the past. As another example, an enterprise may decide to implement a policy that prohibits employees from receiving emails originating from a given domain. Malicious emails that are caught by the anti-spam filter 104 can be quarantined so as to remain hidden from the intended recipients, while non-malicious emails may be stored on an email server 106 for subsequent access by the intended recipients. Email servers (also referred to as “mail servers”) facilitate the delivery of emails from senders to recipients. Normally, an email will be transferred amongst a series of email servers as it travels toward its intended destination. This series of email servers allows emails to be sent between dissimilar address domains.

Because of the manner in which anti-spam filters are deployed, however, these filters struggle to handle graymail in an appropriate manner. As discussed above, graymail generally is not considered spam by the anti-spam filter 104 (e.g., since those email messages are transmitted by a legitimate source, contain legitimate content, etc.). And, in contrast with malicious emails (for example), which are generally universally unwanted, different users may ascribe varying degrees of value to a particular piece of graymail. Accordingly, new approaches are needed in order to appropriately handle graymail.

II. Threat Detection Platform

FIG. 2 depicts an example of a threat detection platform 200 that is configured to identify email messages that constitute graymail and handle those email messages accordingly. As shown in FIG. 2, the threat detection platform 200 includes a profile generator 202, a training module 204, a monitoring module 206, a scoring module 208, and a remediation module 210. Some embodiments of the threat detection platform 200 include a subset of these components, while other embodiments of the threat detection platform 200 include additional components that are not shown in FIG. 2.

Threat detection platform 200 can acquire data related to digital activities performed with email accounts and then determine, based on an analysis of the data, how to handle graymail in a personalized manner. As shown in FIG. 2, the data can include information related to emails and/or mail filters. Threat detection platform 200 can identify instances of graymail based on an analysis of incoming emails (e.g., the content of the body, the email address of the sender, etc.), metadata accompanying the incoming emails (e.g., information regarding the sender, recipient, origin, time of transmission, etc.), and other suitable data.

Threat detection platform 200 can be implemented, partially or entirely, within an enterprise network 212, a remote computing environment (e.g., through which emails, or information related to those emails, can be routed for analysis), a gateway, or another suitable location. The remote computing environment can belong to, or be managed by, the enterprise or another entity. In some embodiments, threat detection platform 200 is integrated into the enterprise's email system (e.g., at the SEG) as part of an inline deployment. In other embodiments, threat detection platform 200 is integrated into the enterprise's email system via an application programming interface (API) such as the Microsoft Outlook® API. In such embodiments, threat detection platform 200 can obtain email data via the API. Thus, the threat detection platform 200 can supplement and/or supplant other security products employed by the enterprise.

In a first variation, threat detection platform 200 is maintained by a threat service (also referred to herein as a “security service”) that has access to multiple enterprises' data. In this variation, threat detection platform 200 can route data related to incoming email to a computing environment managed by the security service. The computing environment can be, for example, an instance on Amazon Web Services® (AWS). Threat detection platform 200 can maintain one or more databases for each enterprise it services that include, for example, organizational charts (and/or other user/group identifiers/memberships, indicating information such as “Alice is a member of the Engineering group” and “Bob is a member of the Marketing group”), attribute baselines, communication patterns, etc. Additionally or alternatively, threat detection platform 200 can maintain federated databases that are shared among multiple entities. Examples of federated databases include databases specifying vendors and conferences for which graymail may be transmitted. The security service can maintain different instances of threat detection platform 200 for different enterprises, or the security service can maintain a single instance of the threat detection platform 200 for multiple enterprises, as applicable. The data hosted in these instances can be obfuscated, encrypted, hashed, depersonalized (e.g., by removing personal identifying information), or otherwise secured or secreted as applicable. Accordingly, in various embodiments, each instance of threat detection platform 200 is only able to access/process data related to the incoming emails addressed to email accounts associated with the corresponding enterprise(s).

In a second variation, threat detection platform 200 is maintained by the enterprise whose emails are being monitored—either remotely or on premises. In this variation, all relevant data related to incoming emails may be hosted by the enterprise itself, and any information to be shared across multiple enterprises can be transmitted to a computing system maintained by the security service or a third party, as applicable.

As shown in FIG. 2, profile generator 202, training module 204, monitoring module 206, scoring module 208, and remediation module 210 can be part of the threat detection platform 200. Alternatively, these components could be implemented individually or in various combinations. For example, remediation module 210 can be implemented in a remote computing environment to which the threat detection platform 200 is communicatively connected across a network. Threat detection platform 200 can be implemented by a security service on behalf of an enterprise or the enterprise itself. In some embodiments, aspects of threat detection platform 200 are provided by a web-accessible computer program operating on a computer server or a distributed computing system. For example, an individual can be able to interface with the threat detection platform 200 through a web browser executing on an electronic computing device (also referred to as an “electronic device” or “computing device”).

Enterprise network 212 can be a mobile network, wired network, wireless network, or some other communication network (or combination of networks) maintained by the enterprise or an operator on behalf of the enterprise. As noted above, the enterprise can use a security service to examine emails (among other things) to discover possible instances of graymail. The enterprise may grant permission to the security service to monitor the enterprise network 212 by examining emails (e.g., incoming emails or outgoing emails), identifying emails that are representative of graymail, and then performing appropriate remediation actions for those emails. In some embodiments, the enterprise further grants permission to the security service to obtain data regarding other digital activities involving the enterprise (and, more specifically, employees of the enterprise) in order to build a profile that specifies communication patterns, behavioral traits, normal content of emails, etc. For example, threat detection platform 200 may identify the filters created by each employee to infer which incoming emails are representative of graymail and/or which graymail is no longer desired (and thus should be diverted). Such filters may comprise rules manually specified by the user (e.g., by the user explicitly interacting with tools made available by cloud-based email suite 308) and/or may also be inferred based on users' interactions with their mail (e.g., by obtaining from cloud-based email suite 308 log data indicating which messages the user has moved from an inbox to a promotions folder or spam folder, or vice versa) and automatically generating rules for automatically moving messages on behalf of the user in the future (without the user having to manually create such rules).

Threat detection platform 200 can manage one or more databases in which data can be stored. Examples of such data include enterprise data (e.g., email data and mail filter data), remediation policies, communication patterns, behavioral traits, and the like. The data stored in the database(s) can be determined by the threat detection platform 200 (e.g., learned from data available on the enterprise network 212), provided by the enterprise, and/or retrieved from an external database (e.g., associated with LinkedIn® or Microsoft Office 365®) as applicable. Threat detection platform 200 can also store outputs produced by the various modules, including machine- and human-readable information regarding discovered instances of graymail and any remediation actions that were taken.

As shown in FIG. 2, threat detection platform 200 includes a profile generator 202 that is responsible for generating one or more profiles for the enterprise. For example, profile generator 202 can generate a separate profile for each email account associated with an employee of the enterprise based on email data and/or mail filter data. Additionally or alternatively, profiles can be generated for business groups, organizational groups, or the enterprise as a whole. By examining email data and mail filter data, profile generator 202 can discover organizational information (e.g., employees, titles, and hierarchy), employee behavioral traits (e.g., based on historical emails and historical mail filters), normal content of incoming or outgoing emails, communication patterns (e.g., who each employee communicates with internally and externally, when each employee normally communicates), etc. This information can be populated into the profiles so that each profile can be used as a baseline for what constitutes normal activity by the corresponding email account (or group of email accounts).

An example profile includes a number of behavioral traits associated with a given email account. For example, profile generator 202 can determine behavioral traits based on email data and mail filter data obtained from the enterprise network 212. The email data may include information on the senders of past emails received by a given email account, content of those past emails, frequency of those past emails, temporal patterns of those past emails, topics of those past emails, geographical location from which those past emails originated, formatting characteristics (e.g., usage of HTML, fonts, styles, etc.), and more. Thus, profile generator 202 can attempt to build a profile for each email account that represents a model of normal behavior of the corresponding employee. As further discussed below, the profiles can be helpful in identifying the emails that are likely representative of graymail, as well as establishing how each employee handles graymail (including different types of graymail).

Monitoring module 206 is responsible for monitoring emails handled by enterprise network 212. These emails can include both incoming emails (e.g., external and internal emails) received by email accounts associated with employees of the enterprise and outgoing emails (e.g., external and internal emails) transmitted by those email accounts. Monitoring module 206 is able to monitor incoming emails in near real time so that appropriate action can be taken, in a timely fashion, if graymail is discovered. For example, if an incoming email is determined to be representative of graymail (e.g., based on an output produced by scoring module 208), the incoming email can be transferred into a dedicated folder by remediation module 210. In some embodiments, monitoring module 206 is able to monitor emails only upon threat detection platform 200 being granted permission by the enterprise (and thus given access to enterprise network 212).

Scoring module 208 can be responsible for examining emails to determine the likelihood that each email is representative of graymail. For example, scoring module 208 can examine each incoming email to determine how its characteristics compare to past emails received by the intended recipient. In such embodiments, scoring module 208 may determine whether characteristics such as timing, formatting, and location of origination (e.g., in terms of sender email address or geographical location) match a pattern of past emails that have been determined to represent graymail. For example, scoring module 208 may determine that an email is highly likely to be graymail if its formatting and content are similar to past emails received at a consistent periodic basis (e.g., daily or weekly).

Scoring module 208 can make use of heuristics, rules, neural networks, or other trained machine learning (ML) approaches such as decision trees (e.g., gradient-boosted decision trees), logistic regression, and linear regression. Accordingly, scoring module 208 can output discrete outputs or continuous outputs, such as a probability metric (e.g., specifying the likelihood that an incoming email is graymail), a binary output (e.g., graymail or not graymail), or a sub-classification (e.g., specifying the type of graymail such as promotions, newsletters, events, and cold calls).

Remediation module 210 can perform one or more remediation actions in response to scoring module 208 determining that an incoming email is likely representative of graymail. The remediation action(s) can be based on whether past instances of graymail have been handled for the same employee, the nature of the graymail, the policies implemented by the enterprise or employee, etc. These policies can be predefined or dynamically generated based on inference, analysis, or the data obtained from enterprise network 212. Additionally or alternatively, remediation action(s) may be based on the outputs produced by the models employed by the various modules, as further discussed below. Examples of remediation actions include creating a graymail folder into which emails that are representative of graymail can be transferred, transferring emails into a graymail folder, and/or transferring emails into another folder such as a quarantine folder. Generally, the graymail folder is accessible through a mail client as other folders, such as sent folders, draft folders, spam folders, etc. Accordingly, while remediation module 210 may redirect graymail before it would otherwise populate into the inbox of the intended recipient, the remediation module 210 may not make graymail inaccessible to the recipient. Stated another way, the transfer of graymail into dedicated folders can be used to declutter the inboxes of employees of the enterprise. Conversely, some graymail (e.g., those emails that may represent a threat) may be transferred to a hidden folder (also referred to as a “quarantine folder”) for further analysis. Emails transferred to the hidden folder may remain inaccessible until the threat detection platform 200 has determined whether to release those emails (e.g., into the inbox or graymail folder), or other applicable event or set of events has occurred to either release it from quarantine (e.g., into an inbox or other folder) or delete it (e.g., if it is determined to represent a threat).

In some embodiments, remediation module 210 provides results produced by scoring module 208 or some other output (e.g., a notification summarizing the graymail that has been found) to an electronic device 214. Electronic device 214 may be managed by the employee associated with the email account under examination, an individual associated with the enterprise (e.g., a member of the information technology department), an individual associated with a security service, etc. In some embodiments, remediation module 210 sends the output in a human-readable format for display on an interface accessible via the electronic device 214. As an example, remediation module 210 can generate a summary of emails that were transferred to the graymail folder. This summary can be provided to the employee to whom these emails were addressed. Through electronic device 214, the employee can specify whether the appropriate action was taken. For instance, the employee may indicate that an email should not have been classified as graymail, or the employee may be able to indicate that an email should have been instead classified as spam. Such indications can be used to improve the treatment of messages sent to that employee in the future (or, as applicable, sent to others, such as sent to other members of the same organizational unit/group, enterprise, etc.).

Various embodiments of threat detection platform 200 include a training module 204 that operates to train the models employed by the other modules. As an example, training module 204 may train the models applied by scoring module 208 to the email data and mail filter data by feeding training data into those models. The training data could include emails that have been labeled as attacks or non-attacks, policies related to attributes of emails (e.g., specifying that emails originating from certain domains should not be considered graymail), etc. The training data may be employee-, group-, or enterprise-specific so that the model(s) are able to perform personalized analysis. In some embodiments, the training data ingested by the model(s) includes emails that are known to be representative of graymail. These emails may have been labeled as such during a training process, or these emails may have been labeled as such by other employees.

A. Graymail Discovery, Classification, and Remediation

FIG. 3 illustrates an example of how a remediation module can implement a graymail remediation service. Graymail remediation service 302 can move email representative of graymail to a dedicated folder (or a folder in a set of dedicated graymail folders). As discussed above, one aspect of graymail remediation service 302 is to create the dedicated folder(s) if one does not already exist for the intended recipient of graymail. Emails can be moved responsive to receiving a request to do so from remediation module 210 based on output(s) produced by a scoring module, or responsive to receiving a request to do so directly from the scoring module (208) as shown in FIG. 3.

Generally, remediation module 210 interacts with two forms of storage while implementing graymail remediation services. First, remediation module 210 may interact with an object-relational-mapping (ORM) model 304 for recording actions performed by graymail remediation service 302. ORM model 304 can create objects that map to relational data that define actions taken. Second, remediation module 210 may interact with a memory cache (also referred to herein as a “cache”) 306 that stores a mapping of employee identifiers to folders and accompanying metadata. As an example, cache 306 can associate email accounts of employees of an enterprise with the folders that can be found in those employees' email accounts. The cache can be read by remediation module 210 in order to find each account's graymail folder(s). If no graymail folder exists and one is created by the remediation module, then the remediation module can update the cached state to indicate that a graymail folder was created.

Though training of scoring module 208 and remediation module 210 may be supervised, graymail remediation service 302 can be implemented in an entirely automated manner. Thus, in various embodiments, remediation module 210 may not require any input from the employees or enterprise whose emails are being monitored.

FIG. 4 illustrates an example of how a graymail remediation service can be implemented in a scalable manner. In the embodiment shown in FIG. 4, scoring module 402 has a separate “pipeline” (404) for emails representative of graymail that (i) persists significantly less data to the ORM model (408) and (ii) directly invokes graymail remediation service 406 (e.g., rather than the general remediation service implemented by the threat detection platform to address email-based threats). The scoring module can detect emails that are representative of graymail and then perform appropriate follow-up action(s). For example, for a sample collection of graymail, scoring module 402 can send those email messages to ORM model 408 for further review. Additionally or alternatively, scoring module 402 can send all email messages determined to be graymail to remediation module 410 for handling by graymail remediation service 406.

In various embodiments, a small collection of email messages are labeled to measure live performance of graymail remediation service 406. These email messages can be stored fully in ORM model 408 so that full analysis can be performed. In other embodiments, ORM model 408 maintains one or more data structures (e.g., tables) in which information regarding graymail can be stored. For example, remediation module 410 may transmit a log of email messages moved to graymail folder 412 (also referred to as a “promotions folder”) so that ORM model 408 includes a data structure that reflects the results of graymail remediation. In some embodiments, more detailed information regarding the graymail is stored in ORM model 408. For example, as shown in FIG. 4, metrics (e.g., scores indicating probability of being graymail) for those email messages determined to be graymail can be populated into the ORM model. Thus, the data populated into the data structure(s) of ORM model 408 can be filtered to include only data relevant to those email messages determined to represent graymail.

Since it is dedicated to remediating graymail, graymail remediation service 406 can be implemented in a less resource-intensive manner than a service for addressing a broad variety of security threats. Nonetheless, graymail remediation service 406 can provide various (including all) of the following guarantees in various embodiments:

    • Guarantee that incoming email messages are promptly moved to the promotions folder (or other appropriate folder, such as “conferences”) when classified as graymail;
    • Guarantee that if graymail remediation service 406 receives multiple requests to move an email message, the email message is only moved once;
    • Guarantee that a promotions folder can be created for an email account if one does not already exist;
    • Guarantee that graymail remediation service 406 does not remediate email messages found in a custom folder;
    • Guarantee that graymail remediation service 406 does not remediate email messages caught by mail filters created by employees or the enterprise; and
    • Guarantee (if such behavior is desired/configured by an administrator or user, for example) that email messages representative of graymail will not be moved by graymail remediation service 406 if the promotions folder is deleted, and that the graymail remediation service will not attempt to recreate the promotions folder (e.g., because the user has implicitly indicated by deleting the promotions folder that the user does not wish to participate in graymail filtering). Note that the number of guarantees may depend on the amount of resources available to the remediation module (and the threat detection platform as a whole) and the amount of insight into incoming emails that is desired.

In some embodiments, threat detection platform 400 tracks how email messages moved to promotions folder 412 are subsequently handled by a user. For example, threat detection platform 400 can employ an ML approach that tracks whether email messages moved to the promotions folder by graymail remediation service 406 are subsequently deleted by the recipient or moved to another folder (e.g., an inbox, or other folder, such as “online shopping” or “travel deals”) by the user. The insights gained by this ML approach can be used in further training the remediation module to identify graymail and also in automatically handling future received messages differently (e.g., when subsequent graymail of a particular type is received, moving it to the user's custom folder, such as “travel deals”).

Moreover, threat detection platform 400 can maintain a list of email addresses corresponding to employees that have explicitly or implicitly opted out of graymail remediation services. For example, threat detection platform 400 can maintain a list of email addresses corresponding to employees who have deleted or renamed the promotions folders made for them by the remediation module. This information may be useful to the enterprise, for example, to identify those employees who have opted not to have graymail automatically filtered. Further follow up (e.g., from the enterprise or a security service) may indicate that these employees find too many non-graymail emails have been transferred to the promotions folder, or that these employees prefer the mail filters that have been manually created to capture graymail originating from certain sources, etc.

B. Managing Graymail at Scale

Many individuals will receive upwards of one hundred times more graymail than emails related to sophisticated attacks (e.g., phishing). A threat detection platform could store the same amount of data per graymail message as per “attack” message discovered by monitoring inbound email. However, such an approach could result in too much data being stored. Because the storage system (also referred to herein as “storage infrastructure”) is shared across different services supported by the threat detection platform, overloading the storage system could impact production of the threat detection platform as a whole. As an example, making too many email messages available for labeling (and training/model creation) may result in overflow of the queue for review. To handle data at such scales, in various embodiments, threat detection platform 400 includes, or has access to, a tiered storage system (“tiered data persistence”) in which graymail occupies only a fraction of the total storage space. Moreover, graymail can be processed by distinct lightweight modules (e.g., those described above with reference to FIG. 2) that allow compute to be scaled efficiently.

An example way of implementing tiered data persistence is to control the percentage of graymail for which data is stored in a tiered storage system (e.g., for use as training data, for use in verification of system reliability, etc.). As an example, threat detection platform 400 can store minimal information (e.g., only that information needed for identification purposes) and metadata for most graymail, and threat detection platform 400 can store complete information for a small subset of the rest of the graymail. An example goal is to store complete information for less than 1, 3, or 5 percent of all graymail. In some embodiments, an administrator or other appropriate individual is able to specify the applicable percentage through an interface provided by the threat detection platform (e.g., an administrative web frontend). Additionally or alternatively, threat detection platform 400 can automatically determine and/or manage the percentage based on predetermined parameters, such as the amount of available computing resources, and/or the rate at which graymail is being received.

In various embodiments, threat detection platform 400 implements a dedicated series of interconnected modules (referred to collectively as a “pipeline”) for handling only graymail. Assume, for example, that threat detection platform 400 includes a scoring module that is responsible for making an initial determination as to whether each incoming email message should be classified as safe, unsafe (i.e., representative of an attack), or graymail. Threat detection platform 400 can include logic to ensure that the module(s) responsible for subsequently handling emails determined to be attacks and the module(s) responsible for subsequently handling graymail do not operate on the same message. The former can be referred to as “attack modules,” while the latter can be referred to as “graymail modules.” While an initial verdict of whether a given email is in fact unsafe may change due to subsequent analysis (e.g., by a human or machine), an initial classification of a message as graymail can be treated (e.g., by graymail modules) as final, because graymail is, in large part, readily confirmable. As such, simplifications can be made that are not possible with the attack modules. Example benefits of using a dedicated graymail pipeline include:

    • As analysis of attacks and graymail are largely decoupled, they can be scaled independently;
    • Storage and processing costs can be kept low despite the large amounts of graymail needing to be handled.

C. Passive Mode for Graymail Discovery Service

An enterprise might desire to initially observe how threat detection platform 400 will handle messages in accordance with techniques described herein before fully implementing graymail handling services (e.g., during a trial period of days, weeks, or months). During that time, threat detection platform 400 can detect and report graymail (and actions that would have been otherwise taken) without actually moving those email messages or otherwise changing recipients' mailboxes. Because the threat detection platform is passively monitoring incoming email messages without impeding those email messages from reaching the intended destination, this mode can also be referred to as “passive mode” for the graymail services.

One benefit of passive mode is that it permits entities to experience a risk-free trial during which confidence in the graymail discovery service can be established. Entities can observe/confirm whether graymail services are behaving as intended (e.g., by identifying graymail, or as applicable, particular subcategories of graymail that should be filtered from recipients' inboxes). A second benefit of passive mode is that it allows adjustments to be made to threat detection platform 400 (e.g., adjustments to be made to settings/configurations of graymail remediation service 406) to adjust performance as necessary. As an example, tuning can be performed for each entity whose email messages are being monitored to account for differences (e.g., in the senders, content, or relevance of incoming messages) between entities. Since the email landscape is different for each entity, this trial period allows changes to be learned by, or implemented in, models employed by the threat detection platform. In some embodiments, during passive mode, employees are encouraged to forward examples of graymail that they receive to dedicated training data collection email addresses (e.g., newsletters@examplecompany.com or events@examplecompany.com) to help customize/tune models/heuristics more specifically to that organization/its users. Further, either during and/or after operating in passive mode, employees can be encouraged to manually move graymail from their inboxes to a graymail folder, or from a spam folder to their inboxes (or a graymail folder), etc. Such user actions, observed by a threat detection platform (e.g., using API calls/log data provided by cloud-based email suite 308) can be used by the threat detection platform to fine-tune graymail handling based on individual preferences. As an example, a first employee may wish to send all airline-related promotions to a graymail folder, while a second employee may wish to send the same messages to a spam folder (or some to a spam folder and some to an inbox, etc.). The threat detection platform can automatically generate different rules for future handling of such messages on behalf of the two different users' actions/preferences.

D. Example Processing System

FIG. 5 is a block diagram illustrating an example of a processing system 500 in which at least some operations described herein can be implemented. For example, components of processing system 500 can be hosted on one or more electronic devices used to provide a threat detection platform (e.g., threat detection platform 200 of FIG. 2).

Processing system 500 includes a processor 502, main memory 506, non-volatile memory 510, network adapter 512 (e.g., a network interface), video display 518, input/output device 520, control device 522 (e.g., a keyboard, pointing device, or mechanical input such as a button), drive unit 524 that includes a storage medium 526, or signal generation device 530 that are communicatively connected to a bus 516. Bus 516 is illustrated as an abstraction that represents one or more physical buses and/or point-to-point connections that are connected by appropriate bridges, adapters, or controllers. Bus 516, therefore, can include a system bus, Peripheral Component Interconnect (PCI) bus, PCI-Express bus, HyperTransport bus, Industry Standard Architecture (ISA) bus, Small Computer System Interface (SCSI) bus, Universal Serial Bus (USB), Inter-Integrated Circuit (I2C) bus, and/or a bus compliant with Institute of Electrical and Electronics Engineers (IEEE) Standard 1394, etc.

While main memory 506, non-volatile memory 510, and storage medium 526 are shown to be a single medium, the terms “storage medium” and “machine-readable medium” should be taken to include a single medium or multiple media that store one or more sets of instructions 528. The terms “storage medium” and “machine-readable medium” should also be taken to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by the processing system 500. Further examples of machine- and computer-readable media include recordable-type media such as volatile and non-volatile memory devices 510, removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD-ROMS) and Digital Versatile Disks (DVDs)), cloud-based storage, and transmission-type media such as digital and analog communication links.

In general, the routines executed to implement embodiments described herein can be implemented as part of an operating system or a specific application, component, program, object, module, or sequence of instructions (collectively referred to as “computer programs”). The computer programs typically comprise one or more instructions (e.g., instructions 504, 508, and/or 528) set at various times in various memories and storage devices in an electronic device. When read and executed by processor 502, the instructions cause processing system 500 to perform operations to execute various aspects of techniques described herein.

Network adapter 512 allows processing system 500 to mediate data in a network 514 with an entity that is external to the processing system 500 through any communication protocol supported by the processing system 500 and the external entity. Examples of network adapter 512 include a network adaptor card, a wireless network interface card, a switch, a protocol converter, a gateway, a bridge, a hub, a receiver, a repeater, and/or a transceiver that includes an integrated circuit (e.g., enabling communication over Bluetooth or Wi-Fi), etc.

Techniques introduced here can be implemented using software, firmware, hardware, or a combination of such forms. For example, various aspects can be implemented using special-purpose hardwired (i.e., non-programmable) circuitry in the form of application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), and the like.

E. Example Process

FIG. 6 illustrates an example of a process for managing potential graymail of an organization. In various embodiments, process 600 is performed by a threat detection platform, such as threat detection platform 400. The process begins at 602 when an electronic message store is accessed. As one example, an electronic message store is accessed by threat detection platform 300 when threat detection platform 300 receives an indication (e.g., provided by an API) that cloud-based email suite 308 has received one or more messages that have not yet been processed by threat detection platform 300. Threat detection platform 300 can then obtain access to each of the unprocessed messages (e.g., via an API) for processing. At 604, a determination is made that a first message included in the electronic message store represents graymail. As discussed above, in various embodiments, threat detection platforms (e.g., 300 and/or 400) include scoring modules (e.g., 208 and/or 402) that use various models/heuristics/etc., as applicable, to determine whether the messages that they process are likely to be graymail or not, including by accessing a profile associated with an addressee (e.g., indicating per-user preferences for different types of graymail handling, and applying those preferences in the context of the message being processed). As further discussed above, such determinations can be a probability metric (e.g., a given message is 98% likely to be graymail), binary (e.g., a given message is graymail), or a sub-classification (e.g., a given message is a newsletter). At 606, and in response to determining that the message represents graymail, a remedial action is taken. As discussed above, in various embodiments, threat detection platforms (e.g., 300 and/or 400) include remediation modules (e.g., 210 and/or 410) that provide graymail remediation services (e.g., 302 and/or 406). Examples of actions that can be taken at 606 include determining whether or not the user has a graymail (or set of graymail) folder(s), creating one(s) if not, and moving the graymail from the user's inbox to the applicable graymail folder. Also as discussed above, if the user subsequently takes an action with respect to the message that was moved by the graymail remediation service to the graymail folder, such action can be used to refine future actions taken by the threat detection platform on behalf of the user (e.g., by not moving similar messages to the graymail folder in the future).

Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.

Claims

1. A system, comprising:

a processor configured to: establish, on behalf of an enterprise, a connection with and using an application programming interface (API) to access an electronic message store that includes a series of communications received by an employee of the enterprise; determine that a first message included in the electronic message store represents graymail, including by accessing a profile associated with an addressee of the first message, and including by applying a set of machine learning models trained using a plurality of different types of graymail as ground truth training data wherein the set of machine learning models can collectively identify graymail and further classify the graymail into one or more of a variety of subcategories; take a remedial action in response to determining that the first message represents graymail; and at a time subsequent to when the remedial action is taken, receive an indication that the addressee has taken an action with respect to the first message, and in response to receiving the indication that the addressee has taken the action, update a rule regarding future remedial actions; and
a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1, wherein determining that the first message represents graymail includes determining that the first message comprises a promotional message opted into by a recipient of the first message, including by using a model trained using promotional messages.

3. The system of claim 1, wherein determining that the first message represents graymail includes determining that the first message comprises a newsletter, including by using a model trained using newsletter messages.

4. The system of claim 1, wherein determining that the first message represents graymail includes determining that the first message comprises an event invitation, including by using a model trained using event invitation messages.

5. The system of claim 1, wherein determining that the first message represents graymail includes determining that the first message comprises a cold call, including by using a model trained using cold call messages.

6. The system of claim 1, wherein taking the remedial action includes determining whether a recipient of the first message has an existing graymail folder.

7. The system of claim 6, wherein, in response to determining that the recipient does not have an existing graymail folder, the processor is further configured to cause a graymail folder to be created.

8. The system of claim 1, wherein taking the remedial action includes causing the first message to be moved from an inbox of a recipient of the first message to a graymail folder of the recipient.

9. The system of claim 1, wherein a first copy of the first message is addressed to a first recipient, wherein a second copy of the first message is addressed to a second recipient that is different from the first recipient, wherein taking the remedial action includes moving the first copy from an inbox of the first recipient, and wherein taking the remedial action further includes not moving the second copy from an inbox of the second recipient.

10. The system of claim 1, wherein the processor is configured to take the remedial action after a period of operating in a passive mode has concluded.

11. The system of claim 10, wherein during the period of operating in passive mode, one or more refinements to a set of remedial actions to be taken in response to determining that messages represent graymail are automatically made.

12. A method, comprising:

establishing, on behalf of an enterprise, a connection with and using an application programming interface (API) to access an electronic message store that includes a series of communications received by an employee of an enterprise;
determining that a first message included in the electronic message store represents graymail, including by accessing a profile associated with an addressee of the first message, and including by applying a set of machine learning models trained using a plurality of different types of graymail as ground truth training data, wherein the set of machine learning models can collectively identify graymail and further classify the graymail into one or more of a variety of subcategories;
taking a remedial action in response to determining that the first message represents graymail; and
at a time subsequent to when the remedial action is taken, receiving an indication that the addressee has taken an action with respect to the first message, and in response to receiving the indication that the addressee has taken the action, updating a rule regarding future remedial actions.

13. The method of claim 12, wherein determining that the first message represents graymail includes determining that the first message comprises a promotional message opted into by a recipient of the first message, including by using a model trained using promotional messages.

14. The method of claim 12, wherein determining that the first message represents graymail includes determining that the first message comprises a newsletter, including by using a model trained using newsletter messages.

15. The method of claim 12, wherein determining that the first message represents graymail includes determining that the first message comprises an event invitation, including by using a model trained using event invitation messages.

16. The method of claim 12, wherein determining that the first message represents graymail includes determining that the first message comprises a cold call, including by using a model trained using cold call messages.

17. The method of claim 12, wherein taking the remedial action includes determining whether a recipient of the first message has an existing graymail folder.

18. The method of claim 17, further comprising, in response to determining that the recipient does not have an existing graymail folder, causing a graymail folder to be created.

19. The method of claim 12, wherein taking the remedial action includes causing the first message to be moved from an inbox of a recipient of the first message to a graymail folder of the recipient.

20. The method of claim 12, wherein a first copy of the first message is addressed to a first recipient, wherein a second copy of the first message is addressed to a second recipient that is different from the first recipient, wherein taking the remedial action includes moving the first copy from an inbox of the first recipient, and wherein taking the remedial action further includes not moving the second copy from an inbox of the second recipient.

21. The method of claim 12, wherein the remedial action is taken after a period of operating in a passive mode has concluded.

22. The method of claim 21, wherein during the period of operating in passive mode, one or more refinements to a set of remedial actions to be taken in response to determining that messages represent graymail are automatically made.

23. A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:

establishing, on behalf of an enterprise, a connection with and using an application programming interface (API) to access an electronic message store that includes a series of communications received by an employee of an enterprise;
determining that a first message included in the electronic message store represents graymail, including by accessing a profile associated with an addressee of the first message, and including by applying a set of machine learning models trained using a plurality of different types of graymail as ground truth training data, wherein the set of machine learning models can collectively identify graymail and further classify the graymail into one or more of a variety of subcategories;
taking a remedial action in response to determining that the first message represents graymail; and
at a time subsequent to when the remedial action is taken, receiving an indication that the addressee has taken an action with respect to the first message, and in response to receiving the indication that the addressee has taken the action, updating a rule regarding future remedial actions.
Referenced Cited
U.S. Patent Documents
5999932 December 7, 1999 Paul
6023723 February 8, 2000 McCormick
6088717 July 11, 2000 Reed
6161130 December 12, 2000 Horvitz
6816884 November 9, 2004 Summers
7263506 August 28, 2007 Lee
7451487 November 11, 2008 Oliver
7546351 June 9, 2009 Horstmann
7577709 August 18, 2009 Kolcz
7610344 October 27, 2009 Mehr
7953814 May 31, 2011 Chasin
8112484 February 7, 2012 Sharma
8131655 March 6, 2012 Cosoi
8191138 May 29, 2012 Emigh
8214437 July 3, 2012 Alspector
8244532 August 14, 2012 Begeja
8566938 October 22, 2013 Prakash
8805934 August 12, 2014 Dargahi
8819819 August 26, 2014 Johnston
8935788 January 13, 2015 Diao
9009824 April 14, 2015 Chen
9021560 April 28, 2015 Emigh
9154514 October 6, 2015 Prakash
9177293 November 3, 2015 Gagnon
9213827 December 15, 2015 Li
9245115 January 26, 2016 Jakobsson
9245225 January 26, 2016 Winn
9264418 February 16, 2016 Crosley
9348981 May 24, 2016 Hearn
9473437 October 18, 2016 Jakobsson
9516053 December 6, 2016 Muddu
9537880 January 3, 2017 Jones
9571512 February 14, 2017 Ray
9686308 June 20, 2017 Srivastava
9756007 September 5, 2017 Stringhini
9774626 September 26, 2017 Himler
9781152 October 3, 2017 Mistratov
9847973 December 19, 2017 Jakobsson
9940394 April 10, 2018 Grant
9946789 April 17, 2018 Li
9954805 April 24, 2018 Nigam
9961096 May 1, 2018 Pierce
9967268 May 8, 2018 Hewitt
10015182 July 3, 2018 Shintre
10044745 August 7, 2018 Jones
10091312 October 2, 2018 Khanwalkar
10104029 October 16, 2018 Chambers
10129194 November 13, 2018 Jakobsson
10129288 November 13, 2018 Xie
10243989 March 26, 2019 Ding
10250624 April 2, 2019 Mixer
10277628 April 30, 2019 Jakobsson
10362057 July 23, 2019 Wu
10397272 August 27, 2019 Bruss
10419468 September 17, 2019 Glatfelter
10523609 December 31, 2019 Subramanian
10601865 March 24, 2020 Mesdaq
10616272 April 7, 2020 Chambers
10673880 June 2, 2020 Pratt
10721195 July 21, 2020 Jakobsson
10749818 August 18, 2020 Bikumala
10834127 November 10, 2020 Yeh
10911489 February 2, 2021 Chechik
10972483 April 6, 2021 Thomas
10972485 April 6, 2021 Ladnai
11019076 May 25, 2021 Jakobsson
11451576 September 20, 2022 Kao
11494421 November 8, 2022 Ghafourifar
11528242 December 13, 2022 Habal
20020002520 January 3, 2002 Gatto
20020116463 August 22, 2002 Hart
20020147988 October 10, 2002 Nakano
20020161782 October 31, 2002 Bozionek
20030009698 January 9, 2003 Lindeman
20030050988 March 13, 2003 Kucherawy
20030204569 October 30, 2003 Andrews
20030212745 November 13, 2003 Caughey
20030224760 December 4, 2003 Day
20040003283 January 1, 2004 Goodman
20040024719 February 5, 2004 Adar
20040030913 February 12, 2004 Liang
20040039786 February 26, 2004 Horvitz
20040083270 April 29, 2004 Heckerman
20040088359 May 6, 2004 Simpson
20040117450 June 17, 2004 Campbell
20040128355 July 1, 2004 Chao
20040167964 August 26, 2004 Rounthwaite
20040186848 September 23, 2004 Kobashikawa
20040215977 October 28, 2004 Goodman
20040260922 December 23, 2004 Goodman
20040267557 December 30, 2004 Liu
20040267886 December 30, 2004 Malik
20050039019 February 17, 2005 Delany
20050044423 February 24, 2005 Mellmer
20050055414 March 10, 2005 Laakkonen
20050076240 April 7, 2005 Appleman
20050101306 May 12, 2005 Zabawskyj
20050102366 May 12, 2005 Kirsch
20050165895 July 28, 2005 Rajan
20050187934 August 25, 2005 Motsinger
20050198518 September 8, 2005 Kogan
20050204006 September 15, 2005 Purcell
20050288961 December 29, 2005 Tabrizi
20060036698 February 16, 2006 Hebert
20060053203 March 9, 2006 Mijatovic
20060161989 July 20, 2006 Reshef
20060168024 July 27, 2006 Mehr
20060191012 August 24, 2006 Banzhof
20060253581 November 9, 2006 Dixon
20060277259 December 7, 2006 Murphy
20070016613 January 18, 2007 Foresti
20070038705 February 15, 2007 Chickering
20070074169 March 29, 2007 Chess
20070112830 May 17, 2007 Danas
20070276851 November 29, 2007 Friedlander
20080005249 January 3, 2008 Hart
20080052398 February 28, 2008 Elshishiny
20080059586 March 6, 2008 Keohane
20080059590 March 6, 2008 Sarafijanovic
20080086532 April 10, 2008 Cunningham
20080114684 May 15, 2008 Foster
20080127345 May 29, 2008 Holtmanns
20080133526 June 5, 2008 Haitani
20080147669 June 19, 2008 Liu
20080162651 July 3, 2008 Madnani
20080172468 July 17, 2008 Almeida
20080201401 August 21, 2008 Pugh
20080294730 November 27, 2008 Oral
20080319932 December 25, 2008 Yih
20090037350 February 5, 2009 Rudat
20090100073 April 16, 2009 Dargahi
20090125528 May 14, 2009 Choi
20090132490 May 21, 2009 Okraglik
20090149203 June 11, 2009 Backholm
20090181651 July 16, 2009 Klassen
20090287618 November 19, 2009 Weinberger
20100017476 January 21, 2010 Shue
20100036786 February 11, 2010 Pujara
20100115040 May 6, 2010 Sargent
20100145900 June 10, 2010 Zheng
20100211641 August 19, 2010 Yih
20100223349 September 2, 2010 Thorson
20100250579 September 30, 2010 Levow
20100318614 December 16, 2010 Sager
20110035451 February 10, 2011 Smith
20110173142 July 14, 2011 Dasgupta
20110179126 July 21, 2011 Wetherell
20110213869 September 1, 2011 Korsunsky
20110214157 September 1, 2011 Korsunsky
20110231510 September 22, 2011 Korsunsky
20110231564 September 22, 2011 Korsunsky
20110238855 September 29, 2011 Korsunsky
20110282948 November 17, 2011 Vitaldevara
20110282954 November 17, 2011 Flake
20120028606 February 2, 2012 Bobotek
20120042017 February 16, 2012 Fried
20120110672 May 3, 2012 Judge
20120137367 May 31, 2012 Dupont
20120143962 June 7, 2012 Bank
20120191716 July 26, 2012 Omoigui
20120215861 August 23, 2012 Smith
20120233662 September 13, 2012 Scott-Cowley
20120253924 October 4, 2012 Giese
20120278887 November 1, 2012 Krish
20120290712 November 15, 2012 Walter
20120297484 November 22, 2012 Srivastava
20130024910 January 24, 2013 Verma
20130041955 February 14, 2013 Chasin
20130086180 April 4, 2013 Midgen
20130086261 April 4, 2013 Lim
20130097709 April 18, 2013 Basavapatna
20130167207 June 27, 2013 Davis
20130191469 July 25, 2013 Dichiu
20130191759 July 25, 2013 Bhogal
20130304742 November 14, 2013 Roman
20140013441 January 9, 2014 Hencke
20140032589 January 30, 2014 Styler
20140181223 June 26, 2014 Homsany
20140325662 October 30, 2014 Foster
20140365303 December 11, 2014 Vaithilingam
20140379825 December 25, 2014 Speier
20140380478 December 25, 2014 Canning
20150026027 January 22, 2015 Priess
20150128274 May 7, 2015 Giokas
20150143456 May 21, 2015 Raleigh
20150149921 May 28, 2015 Hariharan
20150161609 June 11, 2015 Christner
20150161611 June 11, 2015 Duke
20150188862 July 2, 2015 Ghafourifar
20150228004 August 13, 2015 Bednarek
20150234831 August 20, 2015 Prasanna Kumar
20150237068 August 20, 2015 Sandke
20150244657 August 27, 2015 Ghafourifar
20150295942 October 15, 2015 Tao
20150295945 October 15, 2015 Canzanese, Jr.
20150319157 November 5, 2015 Sherman
20150339477 November 26, 2015 Abrams
20150381544 December 31, 2015 Geva
20160014151 January 14, 2016 Prakash
20160036829 February 4, 2016 Sadeh-Koniecpol
20160057167 February 25, 2016 Bach
20160063277 March 3, 2016 Vu
20160072749 March 10, 2016 Lu
20160156654 June 2, 2016 Chasin
20160227367 August 4, 2016 Alsehly
20160253598 September 1, 2016 Yamada
20160262128 September 8, 2016 Hailpern
20160301705 October 13, 2016 Higbee
20160306812 October 20, 2016 McHenry
20160321243 November 3, 2016 Walia
20160328526 November 10, 2016 Park
20160344770 November 24, 2016 Verma
20160380936 December 29, 2016 Gunasekara
20170041296 February 9, 2017 Ford
20170048273 February 16, 2017 Bach
20170063869 March 2, 2017 Treleaven
20170098219 April 6, 2017 Peram
20170111506 April 20, 2017 Strong
20170142056 May 18, 2017 Ganin
20170186112 June 29, 2017 Polapala
20170214701 July 27, 2017 Hasan
20170222960 August 3, 2017 Agarwal
20170223046 August 3, 2017 Singh
20170230323 August 10, 2017 Jakobsson
20170230403 August 10, 2017 Kennedy
20170237754 August 17, 2017 Todorovic
20170237776 August 17, 2017 Higbee
20170251006 August 31, 2017 Larosa
20170289191 October 5, 2017 Thioux
20170324689 November 9, 2017 Baek
20170324767 November 9, 2017 Srivastava
20170346853 November 30, 2017 Wyatt
20180026926 January 25, 2018 Nigam
20180027006 January 25, 2018 Zimmermann
20180084003 March 22, 2018 Uriel
20180084013 March 22, 2018 Dalton
20180091453 March 29, 2018 Jakobsson
20180091476 March 29, 2018 Jakobsson
20180109480 April 19, 2018 Syrowitz
20180131652 May 10, 2018 Smith
20180159808 June 7, 2018 Pal
20180188896 July 5, 2018 Ghafourifar
20180189347 July 5, 2018 Ghafourifar
20180196942 July 12, 2018 Kashyap
20180205691 July 19, 2018 Osipkov
20180219817 August 2, 2018 Zang
20180219823 August 2, 2018 Mohan
20180219888 August 2, 2018 Apostolopoulos
20180227324 August 9, 2018 Chambers
20180232441 August 16, 2018 Lin
20180295146 October 11, 2018 Kovega
20180324297 November 8, 2018 Kent
20180375814 December 27, 2018 Hart
20190014143 January 10, 2019 Syme
20190026461 January 24, 2019 Cidon
20190028509 January 24, 2019 Cidon
20190052655 February 14, 2019 Benishti
20190065748 February 28, 2019 Foster
20190068616 February 28, 2019 Woods
20190081983 March 14, 2019 Teal
20190087428 March 21, 2019 Crudele
20190089711 March 21, 2019 Faulkner
20190104154 April 4, 2019 Kumar et al.
20190109863 April 11, 2019 Traore
20190141183 May 9, 2019 Chandrasekaran
20190166161 May 30, 2019 Anand
20190166162 May 30, 2019 Anand
20190190929 June 20, 2019 Thomas
20190190936 June 20, 2019 Thomas
20190199745 June 27, 2019 Jakobsson
20190205511 July 4, 2019 Zhan
20190222606 July 18, 2019 Schweighauser
20190238571 August 1, 2019 Adir
20190260780 August 22, 2019 Matthew
20190311121 October 10, 2019 Martin
20190319905 October 17, 2019 Baggett
20190319987 October 17, 2019 Levy
20190349400 November 14, 2019 Bruss
20190384911 December 19, 2019 Caspi
20200007502 January 2, 2020 Everton
20200021609 January 16, 2020 Kuppanna
20200034752 January 30, 2020 Luo
20200044851 February 6, 2020 Everson
20200053111 February 13, 2020 Jakobsson
20200053120 February 13, 2020 Wilcox
20200067861 February 27, 2020 Leddy
20200068031 February 27, 2020 Kursun
20200074078 March 5, 2020 Saxe
20200076825 March 5, 2020 Vallur
20200097911 March 26, 2020 Padmanaban
20200125725 April 23, 2020 Petersen
20200127962 April 23, 2020 Chuhadar
20200162483 May 21, 2020 Farhady
20200204572 June 25, 2020 Jeyakumar
20200250527 August 6, 2020 Zhao
20200287936 September 10, 2020 Nguyen
20200344251 October 29, 2020 Jeyakumar
20200358804 November 12, 2020 Crabtree
20200374251 November 26, 2020 Warshaw
20200389486 December 10, 2020 Jeyakumar
20200396190 December 17, 2020 Pickman
20200396258 December 17, 2020 Jeyakumar
20200412767 December 31, 2020 Crabtree
20210021612 January 21, 2021 Higbee
20210058395 February 25, 2021 Jakobsson
20210091962 March 25, 2021 Finke
20210092154 March 25, 2021 Kumar
20210126944 April 29, 2021 Lesperance
20210168161 June 3, 2021 Dunn
20210240836 August 5, 2021 Hazony
20210272066 September 2, 2021 Bratman
20210273950 September 2, 2021 Lawson
20210295179 September 23, 2021 Eyal Altman
20210329035 October 21, 2021 Jeyakumar
20210336983 October 28, 2021 Lee
20210352093 November 11, 2021 Hassanzadeh
20210360027 November 18, 2021 Boyer
20210374679 December 2, 2021 Bratman
20210374680 December 2, 2021 Bratman
20210385181 December 9, 2021 Parkinson
20210400008 December 23, 2021 Khan
20210406836 December 30, 2021 Bar-on
20220021700 January 20, 2022 Devlin
20220131821 April 28, 2022 Habal
20220272062 August 25, 2022 Habal
Foreign Patent Documents
107315954 November 2017 CN
Other references
  • Barngrover, Adam, “Vendor Access Management with IGA”, Saviynt Inc. Apr. 24, 2019 (Apr. 24, 2019) Retrieved on Apr. 17, 2021 (Apr. 17, 2021) from <https://saviynt.com/vendor-access-management-with-iga/> entire document, 2 pp.
  • International Search Report and Written Opinion dated Apr. 24, 2020 of PCT/US2019/067279 (14 pages).
  • Mahajan, et al., “Finding HTML Presentation Failures Using Image Comparison Techniques”, ASE'14, pp. 91-98 (Year: 2014).
  • Mont, Marco Casassa, “Towards accountable management of identity and privacy: Sticky policies and enforceable tracing services”, 14th International Workshop on Database and Expert Systems Applications, 2003. Proceedings. IEEE, 2003. Mar. 19, 2003 (Mar. 19, 2003), Retrieved on Apr. 17, 2021 (Apr. 17, 2021) from <https://ieeexplore.ieee.org/abstract/documenV1232051 > entire document, Mar. 19, 2003, 17 pp.
  • Proofpoint (Proofpoint Closed-Loop Email Analysis and Response, Aug. 2018, 2 pages) (Year: 2018).
  • Information Security Media Group, “Multi-Channel Fraud: A Defense Plan”, Retrieved on Apr. 18, 2021 (Apr. 18, 2021) from <https ://www .bankInfosecurity.com/Interviews/multi-channel-fraud-defense-plan-i-1799>, Feb. 20, 2013, 9 pages.
Patent History
Patent number: 11683284
Type: Grant
Filed: May 12, 2022
Date of Patent: Jun 20, 2023
Patent Publication Number: 20220272062
Assignee: Abnormal Security Corporation (San Francisco, CA)
Inventors: Rami F. Habal (San Francisco, CA), Kevin Lau (San Francisco, CA), Sharan Dev Sankar (Brooklyn, NY), Yea So Jung (San Mateo, CA), Dhruv Purushottam (San Mateo, CA), Venkat Krishnamoorthi (San Jose, CA), Franklin X. Wang (Bellevue, WA), Jeshua Alexis Bratman (San Francisco, CA), Jocelyn Mikael Raphael Beauchesne (San Francisco, CA), Abhijit Bagri (San Francisco, CA), Sanjay Jeyakumar (Berkeley, CA)
Primary Examiner: Backhean Tiv
Application Number: 17/743,048
Classifications
Current U.S. Class: Generating Database Or Data Structure (e.g., Via User Interface) (707/102) (707/999.102)
International Classification: H04L 51/08 (20220101); H04L 51/42 (20220101); H04L 51/212 (20220101); G06Q 10/107 (20230101); G06F 9/54 (20060101);