Patents Assigned to metacluster lt, UAB
-
Publication number: 20230033150Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: ApplicationFiled: September 30, 2022Publication date: February 2, 2023Applicant: Metacluster LT, UABInventors: Eivydas VILCINSKAS, Arnas PETRUSKEVICIUS, Giedrius STALIORAITIS, Martynas JURAVICIUS, Rimantas STANKEVICIUS
-
Publication number: 20230018983Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: ApplicationFiled: July 12, 2021Publication date: January 19, 2023Applicant: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Publication number: 20230017698Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: ApplicationFiled: March 21, 2022Publication date: January 19, 2023Applicant: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Publication number: 20230018387Abstract: The current application discloses processor-implemented methods and systems of processing unclassified HTML responses collected in the context of a data collection service, the method comprising, in one embodiment, receiving unclassified HTML documents, isolating elements relevant for category identification, deriving classification attributes from the isolated elements, and applying a Machine Learning-based classification model resulting in HTML data items classified and labelled accordingly. In certain embodiments the Machine Learning model may be a model trained on a pre-created training data set labeled manually or in an automatic fashion.Type: ApplicationFiled: July 6, 2021Publication date: January 19, 2023Applicant: Metacluster LT, UABInventors: Andrius KUKSTA, Jurijus GORSKOVAS, Martynas JURAVICIUS
-
Publication number: 20230018506Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: ApplicationFiled: April 21, 2022Publication date: January 19, 2023Applicant: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Publication number: 20220414164Abstract: In one aspect, methods and systems for producing an index of a target website are described. In another aspect, methods and systems for extracting specific information from one or more specific indexed URLs are described. The method and system for producing an index of a target website include receiving and analyzing a client's specifications for the index, accessing a target website, extracting the relevant information from the target website, parsing the extracted information in order to identify the URLs, producing the index containing the identified URLs, storing the index (which contains the list of indexed URLs) in a database, compiling the index (which contains the list of indexed URLs) into different formats requested by the client and providing the client, the access information for accessing the compiled index.Type: ApplicationFiled: June 28, 2021Publication date: December 29, 2022Applicant: Metacluster LT, UABInventors: Eivydas VILCINSKAS, Rimantas STANKEVICIUS, Aleksandras SULZENKO
-
Publication number: 20220414166Abstract: ADVANCED RESPONSE PROCESSING IN WEB DATA COLLECTION discloses processor-implemented apparatuses, methods, and systems of processing unstructured raw HTML responses collected in the context of a data collection service, the method comprising, in one embodiment, receiving raw unstructured HTML documents and extracting text data with associated meta information that may comprise style and formatting information. In some embodiments data field tags and values may be assigned to the text blocks extracted, classifying the data based on the processing of Machine Learning algorithms. Additionally, blocks of extracted data may be grouped and re-grouped together and presented as a single data point. In another embodiment the system may aggregate and present the text data with the associated meta information in a structured format. In certain embodiments the Machine Learning model may be a model trained on a pre-created training data set labeled manually or in an automatic fashion.Type: ApplicationFiled: July 1, 2022Publication date: December 29, 2022Applicant: Metacluster LT, UABInventors: Martynas JURAVICIUS, Andrius KUKSTA
-
Patent number: 11416564Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: GrantFiled: July 12, 2021Date of Patent: August 16, 2022Assignee: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Patent number: 11416291Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: GrantFiled: July 12, 2021Date of Patent: August 16, 2022Assignee: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Patent number: 11379542Abstract: ADVANCED RESPONSE PROCESSING IN WEB DATA COLLECTION discloses processor-implemented apparatuses, methods, and systems of processing unstructured raw HTML responses collected in the context of a data collection service, the method comprising, in one embodiment, receiving raw unstructured HTML documents and extracting text data with associated meta information that may comprise style and formatting information. In some embodiments data field tags and values may be assigned to the text blocks extracted, classifying the data based on the processing of Machine Learning algorithms. Additionally, blocks of extracted data may be grouped and re-grouped together and presented as a single data point. In another embodiment the system may aggregate and present the text data with the associated meta information in a structured format. In certain embodiments the Machine Learning model may be a model trained on a pre-created training data set labeled manually or in an automatic fashion.Type: GrantFiled: June 25, 2021Date of Patent: July 5, 2022Assignee: Metacluster LT, UABInventors: Martynas Juravicius, Andrius Kuksta
-
Patent number: 11372937Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: GrantFiled: July 12, 2021Date of Patent: June 28, 2022Assignee: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Publication number: 20220100808Abstract: Systems and methods of task implementation are extended as provided herein and target the web crawling process through a step of submitting a request by a customer to a web crawler. The systems and methods allow a request for a web crawler to be enriched with a customized browsing profile in order to be categorized as an organic human user to obtain targeted content.Type: ApplicationFiled: September 29, 2020Publication date: March 31, 2022Applicant: metacluster lt, UABInventor: Martynas Juravicius
-
Patent number: 11281730Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: GrantFiled: July 12, 2021Date of Patent: March 22, 2022Assignee: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Patent number: 11204971Abstract: Embodiments disclose a system that allows for improved generation of web requests for scraping that, because of the nature of the requests and time and manner they are sent out, appear more organic, as in human generated, than conventional automated scraping systems. The system then manages how a client request to scrape a target website is made to the site, masking the request in a manner that makes it appear to the Web server as if the request is not generated by an automated system. In this way, by appearing more organic, Web servers may be less likely to block requests from the disclosed system or may take longer to block requests from the disclosed system. By avoiding Web servers blocking requests and extending the lifetime of IP proxies before they are blocked, embodiments can use a limited IP proxy address space more efficiently.Type: GrantFiled: July 12, 2021Date of Patent: December 21, 2021Assignee: Metacluster LT, UABInventors: Eivydas Vilcinskas, Arnas Petruskevicius, Giedrius Stalioraitis, Martynas Juravicius, Rimantas Stankevicius
-
Patent number: 10798209Abstract: The task, logic of HTTP/HTTPS session statistics interception and collection is moved to the client side instead of the proxy layer. Encrypted HTTPS tunnel is terminated at the client end, making the actual content or data in transit invisible to both proxies and the smart proxy rotator (SPR). Client's scraping software has a plug-in installed that expands its functionality. HTTP/HTTPS session quality metrics are intercepted and collected at the client side, then sent to the SPR. Proxy usage mark “can be used” is obtained from the SPR for the currently analyzed proxy, based on the results of metrics analysis.Type: GrantFiled: March 19, 2020Date of Patent: October 6, 2020Assignee: metacluster lt, UABInventors: Martynas Juravicius, Eivydas Vilcinskas
-
Patent number: 10601948Abstract: The task, logic of HTTP/HTTPS session statistics interception and collection is moved to the client side instead of the proxy layer. Encrypted HTTPS tunnel is terminated at the client end, making the actual content or data in transit invisible to both proxies and the smart proxy rotator (SPR). Client's scraping software has a plug-in installed that expands its functionality. HTTP/HTTPS session quality metrics are intercepted and collected at the client side, then sent to the SPR. Proxy usage mark “can be used” is obtained from the SPR for the currently analyzed proxy, based on the results of metrics analysis.Type: GrantFiled: October 30, 2019Date of Patent: March 24, 2020Assignee: metacluster lt, UABInventors: Martynas Juravicius, Eivydas Vilcinskas