Abstract: Techniques are disclosed for analyzing documents to detect web components and the web frameworks in the documents. In at least one embodiment, a network analysis system is provided to passively detect web frameworks of documents. The network analysis system can render a document using a document object model to identify objects in the document that are defined as web components. A hash function may be applied to each of the objects to generate a hash signature for the object. Files defining web frameworks can be downloaded from a repository system. Each file may corresponding to a web component. A hash function is applied content in each file to generate a hash signature. The hash signatures of each file may be compared to the hash signatures of the objects in the document to identify a web component for each object. A web framework can be identified based on the web components.

Abstract: An inventory of Internet-facing assets related to a target domain is generated using network data gathered from network data sources. Using data sources of known threats, such as malware, phishing attempts, scam pages, blacklisted sites, and so on, a network analytic system generates analytical information about domains, sub-domains, and components that are owned, managed, and/or controlled by a target entity. A measure of domain identity threat is generated based on a classification model using the analytical information.

Abstract: A method includes receiving a seed at a computing device. The method further includes identifying, based on first domain name system (DNS) data, first border gateway protocol (BGP) data, first whois data, or a combination thereof, a plurality of first internet-facing assets related to the seed. The method further includes identifying, based on second DNS data, second BGP data, second whois data, or a combination thereof, a plurality of second internet-facing assets related to at least one of the first internet-facing assets. The method further includes generating a graphical user interface (GUI) that includes a list of internet-facing assets related to the seed, where the list includes the plurality of first internet-facing assets and the plurality of second internet-facing assets.

Abstract: Embodiments are directed to using a hash signature of a rendered DOM object of a website to find similar content and behavior on other websites. Embodiments break a DOM into a large number of data portions (i.e., “shingles”), apply a hashing algorithm to the shingles, select a predetermined number of hashes from the hashed shingles according to a selection criteria to create a hash signature, and compare the hash signature to that of a reference page to determine similarity of website DOM object content. Embodiments can be used to identify phishing websites, defaced websites, spam websites, significant changes in the content of a webpage, copyright infringement, and any other suitable purposes related to the similarity between website DOM object content.

Abstract: Embodiments of the present invention are directed to identifying phishing websites by rendering and analyzing document object model (DOM) objects associated with a website for features that indicate phishing behavior. Embodiments analyze the full scope and functionality associated with a website by executing functions embedded in a DOM object before analyzing the website for phishing activity. Accordingly, embodiments render and analyze a fully executed DOM object for phishing behavior. Embodiments may then perform steps to mediate a website that is classified as performing phishing. Thus, embodiments are configured to (1) collect website information from a variety of websites and web servers connected to the internet, (2) analyze the collected data to determine whether the website information is performing phishing, and (3) mediate websites and other actors that are determined to be performing phishing based on the results of the phishing analysis.

Abstract: The present disclosure generally relates to web page analysis, and more particularly to detecting malicious behavior using an accomplice model. In certain embodiments, the accomplice model may determine that a URI is associated with malicious behavior based upon the URI being associated with an attribute determined to be related to malicious behavior. Examples of an attribute include a host system, a domain, or an element of a document used to render the web page. Examples of an element of a document used to render the web page may include an active/dynamic element (e.g., a function, a script, etc.) or an inactive/static element (e.g., a string, a number, a frame, a tracking username, a social networking username, etc.).

Abstract: Embodiments of the present invention are directed to identifying phishing websites by rendering and analyzing document object model (DOM) objects associated with a website for features that indicate phishing behavior. Embodiments analyze the full scope and functionality associated with a website by executing functions embedded in a DOM object before analyzing the website for phishing activity. Accordingly, embodiments render and analyze a fully executed DOM object for phishing behavior. Embodiments may then perform steps to mediate a website that is classified as performing phishing. Thus, embodiments are configured to (1) collect website information from a variety of websites and web servers connected to the internet, (2) analyze the collected data to determine whether the website information is performing phishing, and (3) mediate websites and other actors that are determined to be performing phishing based on the results of the phishing analysis.

Abstract: The present disclosure generally relates to web page analysis, and more particularly to detecting malicious behavior using an accomplice model. In certain embodiments, the accomplice model may determine that a URI is associated with malicious behavior based upon the URI being associated with an attribute determined to be related to malicious behavior. Examples of an attribute include a host system, a domain, or an element of a document used to render the web page. Examples of an element of a document used to render the web page may include an active/dynamic element (e.g., a function, a script, etc.) or an inactive/static element (e.g., a string, a number, a frame, a tracking username, a social networking username, etc.).

Abstract: Techniques are disclosed for analyzing documents to detect web components and the web frameworks in the documents. In at least one embodiment, a network analysis system is provided to passively detect web frameworks of documents. The network analysis system can render a document using a document object model to identify objects in the document that are defined as web components. A hash function may be applied to each of the objects to generate a hash signature for the object. Files defining web frameworks can be downloaded from a repository system. Each file may corresponding to a web component. A hash function is applied content in each file to generate a hash signature. The hash signatures of each file may be compared to the hash signatures of the objects in the document to identify a web component for each object. A web framework can be identified based on the web components.

Abstract: Techniques are disclosed for analyzing documents to detect web components and the web frameworks in the documents. In at least one embodiment, a network analysis system is provided to passively detect web frameworks of documents. The network analysis system can render a document using a document object model to identify objects in the document that are defined as web components. A hash function may be applied to each of the objects to generate a hash signature for the object. Files defining web frameworks can be downloaded from a repository system. Each file may corresponding to a web component. A hash function is applied content in each file to generate a hash signature. The hash signatures of each file may be compared to the hash signatures of the objects in the document to identify a web component for each object. A web framework can be identified based on the web components.

Abstract: A method includes receiving a seed at a computing device. The method further includes identifying, based on first domain name system (DNS) data, first border gateway protocol (BGP) data, first whois data, or a combination thereof, a plurality of first internet-facing assets related to the seed. The method further includes identifying, based on second DNS data, second BGP data, second whois data, or a combination thereof, a plurality of second internet-facing assets related to at least one of the first internet-facing assets. The method further includes generating a graphical user interface (GUI) that includes a list of internet-facing assets related to the seed, where the list includes the plurality of first internet-facing assets and the plurality of second internet-facing assets.

Abstract: Embodiments are directed to using a hash signature of a rendered DOM object of a website to find similar content and behavior on other websites. Embodiments break a DOM into a large number of data portions (i.e., “shingles”), apply a hashing algorithm to the shingles, select a predetermined number of hashes from the hashed shingles according to a selection criteria to create a hash signature, and compare the hash signature to that of a reference page to determine similarity of website DOM object content. Embodiments can be used to identify phishing websites, defaced websites, spam websites, significant changes in the content of a webpage, copyright infringement, and any other suitable purposes related to the similarity between website DOM object content.

Abstract: A method includes receiving a seed at a computing device. The method further includes identifying, based on first domain name system (DNS) data, first border gateway protocol (BGP) data, first whois data, or a combination thereof, a plurality of first internet-facing assets related to the seed. The method further includes identifying, based on second DNS data, second BGP data, second whois data, or a combination thereof, a plurality of second internet-facing assets related to at least one of the first internet-facing assets. The method further includes generating a graphical user interface (GUI) that includes a list of internet-facing assets related to the seed, where the list includes the plurality of first internet-facing assets and the plurality of second internet-facing assets.

Abstract: Embodiments are directed to using a hash signature of a rendered DOM object of a website to find similar content and behavior on other websites. Embodiments break a DOM into a large number of data portions (i.e., “shingles”), apply a hashing algorithm to the shingles, select a predetermined number of hashes from the hashed shingles according to a selection criteria to create a hash signature, and compare the hash signature to that of a reference page to determine similarity of website DOM object content. Embodiments can be used to identify phishing websites, defaced websites, spam websites, significant changes in the content of a webpage, copyright infringement, and any other suitable purposes related to the similarity between website DOM object content.

Abstract: The present disclosure generally relates to web page analysis, and more particularly to detecting malicious behavior using an accomplice model. In certain embodiments, the accomplice model may determine that a URI is associated with malicious behavior based upon the URI being associated with an attribute determined to be related to malicious behavior. Examples of an attribute include a host system, a domain, or an element of a document used to render the web page. Examples of an element of a document used to render the web page may include an active/dynamic element (e.g., a function, a script, etc.) or an inactive/static element (e.g., a string, a number, a frame, a tracking username, a social networking username, etc.).

Abstract: The present disclosure generally relates to web page analysis, and more particularly to a classification system for web pages. The classification system may classify a web page as malicious based upon one or more signatures generated for the web page. For example, the classification system may compare one or more signatures generated for a first web page to one or more signatures generated for a second web page, where the first web page and the second web page are the same web page at different times or different web pages. Based upon a similarity of the signatures, the classification system may output whether the first web page is malicious. For another example, the classification system may include a classification model that is trained based upon one or more signatures for one or more classified web pages. The classification model may output whether the web page is malicious.

Abstract: The present disclosure relates to identifying and storing relationships between hosts that are used to present a web page to a user. In certain embodiments, a system for detecting host pairs is provided. The system may receive a first request to identify one or more host pairs associated with a first host. In response to receiving the first request, the system may send a second request to the first host for a document. The document may be a web page file that is used to build a web page associated with the first host. The web page file may include instructions that, when parsed, build the web page. In response to the second request, the first host may send a response to the system. The system may then use the data included in the response to build the web page. While building the web page, a pairing may be stored when a different host is contacted.

Abstract: Embodiments of the present disclosure are directed to a network analytic system for tracking and analysis of network infrastructure for network-based digital assets. The network analytic system can detect and track a relationship between assets based on one or more attributes related or shared between any given assets. The network analytic system can analyze network-based digital assets to determine information about a website (e.g., information about electronic documents, such as web pages) that has be used to detect phishing and other abuse of the website. The network analytic system can analyze data about network-based assets to determine whether any are being used or connected to use of unauthorized or malicious activity or known network-based assets. Based on the relationship identified, the network analytic system can associate or link assets together. The network analytic system may provide an interface to view data sets generated by the network analytic system.

Abstract: Techniques are disclosed for analyzing documents to detect web components and the web frameworks in the documents. In at least one embodiment, a network analysis system is provided to passively detect web frameworks of documents. The network analysis system can render a document using a document object model to identify objects in the document that are defined as web components. A hash function may be applied to each of the objects to generate a hash signature for the object. Files defining web frameworks can be downloaded from a repository system. Each file may corresponding to a web component. A hash function is applied content in each file to generate a hash signature. The hash signatures of each file may be compared to the hash signatures of the objects in the document to identify a web component for each object. A web framework can be identified based on the web components.

Abstract: Embodiments are directed to using a hash signature of a rendered DOM object of a website to find similar content and behavior on other websites. Embodiments break a DOM into a large number of data portions (i.e., “shingles”), apply a hashing algorithm to the shingles, select a predetermined number of hashes from the hashed shingles according to a selection criteria to create a hash signature, and compare the hash signature to that of a reference page to determine similarity of website DOM object content. Embodiments can be used to identify phishing websites, defaced websites, spam websites, significant changes in the content of a webpage, copyright infringement, and any other suitable purposes related to the similarity between website DOM object content.