Patents by Inventor Daniel E. Sotolongo
Daniel E. Sotolongo has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11966416Abstract: Techniques for triggering pipeline execution based on data change (transaction commit) are described. The pipelines can be used for data ingestion or other specified tasks. These tasks can be operational across account, organization, cloud region, and cloud provider boundaries. The tasks can be triggered by commit post-processing. Gates in the tasks can be set up to reference change data capture information. If the gate is satisfied, tasks can be executed to set up data pipelines.Type: GrantFiled: January 31, 2023Date of Patent: April 23, 2024Assignee: Snowflake Inc.Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Dinesh Chandrakant Kulkarni, Daniel Mills, Daniel E. Sotolongo, Di Fei Zhang
-
Publication number: 20240126765Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.Type: ApplicationFiled: December 21, 2023Publication date: April 18, 2024Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Daniel E. Sotolongo, Zhuo Zhang
-
Patent number: 11921700Abstract: Techniques for creating and using error tables to track errors associated with a base table are described. A command to perform an operation on a base table stored in a network-based data system can be received and executed, causing at least one error. At least one error record corresponding to the at least one error can be inputted into an error table, which is nested with the base table. Contextual information can be added to the at least one error record.Type: GrantFiled: May 18, 2023Date of Patent: March 5, 2024Assignee: Snowflake Inc.Inventors: Abdullah Al Mahmood, Tyler Jones, Xin Huang, Ganeshan Ramachandran Iyer, Jiaxing Liang, Daniel Mills, Subramanian Muralidhar, Daniel E. Sotolongo
-
Patent number: 11893029Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.Type: GrantFiled: October 25, 2022Date of Patent: February 6, 2024Assignee: Snowflake Inc.Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Daniel E. Sotolongo, Zhuo Zhang
-
Patent number: 11860850Abstract: Provided herein are systems and methods for a stream object configuration, including configuring and processing streams on views and streams on tables. For example, a method includes parsing, by at least one hardware processor, a received query to determine a plurality of data processing operations associated with a base table. A stream object on the base table is generated. The stream object includes a plurality of query results from executing the plurality of data processing operations and a corresponding plurality of time stamps of times the plurality of query results are obtained. A stream expansion of the stream object is performed. The stream expansion is based on computing changes to the base table from the plurality of data processing operations.Type: GrantFiled: October 24, 2022Date of Patent: January 2, 2024Assignee: Snowflake Inc.Inventors: Istvan Cseri, Tyler Jones, Daniel Mills, Daniel E. Sotolongo
-
Publication number: 20230418807Abstract: Provided herein are systems and methods for a stream object configuration, including query processing of stream objects using stream expansion. For example, a method includes decoding a query to obtain a first data processing operation and a first stream object. The first stream object is associated with a view on a base table. A first stream expansion on the first stream object is performed. The first stream expansion is based on generating a second stream object on the base table. A second stream expansion of the second stream object is performed. The second stream expansion is based on replacing the second stream object with at least a second data processing operation. The query is executed based on completing the first data processing operation and the at least a second data processing operation.Type: ApplicationFiled: August 31, 2023Publication date: December 28, 2023Inventors: Istvan Cseri, Tyler Jones, Daniel Mills, Daniel E. Sotolongo
-
Publication number: 20230409574Abstract: A system for a materialized table (MT) refresh using multiple processing pipelines includes at least one hardware processor coupled to memory storing instructions. The instructions cause the at least one hardware processor to perform operations including determining dependencies among a plurality of intermediate MTs generated from a source MT. The source MT uses a table definition with a query on one or more base tables and a lag duration value. A graph snapshot of dependencies among the plurality of intermediate MTs is generated. Processing pipelines are configured. Each of the processing pipelines corresponds to a subset of the plurality of intermediate MTs indicated by the graph snapshot. Responsive to detecting an instruction for a refresh operation on the source MT, refreshes on corresponding intermediate MTs of the plurality of intermediate MTs in each processing pipeline of the processing pipelines are performed to complete the refresh operation on the source MT.Type: ApplicationFiled: July 31, 2023Publication date: December 21, 2023Inventors: Tyler Arthur Akidau, Fabian Hueske, Tyler Jones, Daniel Mills, Leon Papke, Prasanna Rajaperumal, Daniel E. Sotolongo
-
Publication number: 20230401199Abstract: A method includes retrieving a plurality of materialized tables (MTs). Each of the plurality of MTs includes a lag duration and refers to a corresponding base table of a plurality of base tables. The lag duration indicates a maximum time period that a result of a prior refresh of a query on the corresponding base table can lag behind a current time instance. A plurality of time instances for the MT is determined based on the lag duration and a number of prior refreshes of the corresponding base table. A plurality of aligned time instances for the plurality of MTs is determined based on the plurality of time instances for each of the plurality of MTs. Refresh operations are scheduled for the plurality of MTs at one or more of the plurality of aligned time instances that are within the maximum time period.Type: ApplicationFiled: July 17, 2023Publication date: December 14, 2023Inventors: Tyler Arthur Akidau, Tyler Jones, Daniel Mills, Leon Papke, Prasanna Rajaperumal, Daniel E. Sotolongo
-
Publication number: 20230367757Abstract: A system or persistent table may be generated storing changelog information of a primary base table. The system table may then be used to create streams of relevant information. In some examples, the streams may read from the system table for information past a retention period of the primary table while reading from the primary table information in the retention period.Type: ApplicationFiled: July 26, 2023Publication date: November 16, 2023Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Daniel E. Sotolongo
-
Publication number: 20230342377Abstract: Techniques described herein can enable stream replication. A first deployment can store a table including one or more streams. The techniques described herein can be used to replicate the table at a second deployment while replicating the one or more streams associated with the table. Select prior table versions and partitions in the table are copied to the second deployment to enable stream replication.Type: ApplicationFiled: June 30, 2023Publication date: October 26, 2023Inventors: Istvan Cseri, Tyler Jones, Daniel E. Sotolongo, Boyuan Zhang
-
Publication number: 20230315755Abstract: Techniques for triggering pipeline execution based on data change (transaction commit) are described. The pipelines can be used for data ingestion or other specified tasks. These tasks can be operational across account, organization, cloud region, and cloud provider boundaries. The tasks can be triggered by commit post-processing. Gates in the tasks can be set up to reference change data capture information. If the gate is satisfied, tasks can be executed to set up data pipelines.Type: ApplicationFiled: January 31, 2023Publication date: October 5, 2023Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Dinesh Chandrakant Kulkarni, Daniel Mills, Daniel E. Sotolongo, Di Fei Zhang
-
Patent number: 11762855Abstract: Provided herein are systems and methods for a database object (e.g., materialized table) configuration including using the database object for incremental maintenance of query results. For example, a method includes creating a materialized table using a table definition. The table definition includes a query on one or more base tables and a lag duration value. The lag duration value indicates a maximum time period that a result of a prior refresh of the query can lag behind a current time instance. A delta set for the one or more base tables is determined. The delta set includes at least one change applied to the one or more base tables since the prior refresh of the query. The delta set is merged with the materialized table to perform an incremental update of the materialized table.Type: GrantFiled: September 13, 2022Date of Patent: September 19, 2023Assignee: Snowflake Inc.Inventors: Tyler Arthur Akidau, Fabian Hueske, Tyler Jones, Daniel Mills, Leon Papke, Prasanna Rajaperumal, Daniel E. Sotolongo
-
Patent number: 11755568Abstract: Provided herein are systems and methods for a database object (e.g., materialized table) configuration including scheduling refreshes of the materialized table. For example, a method includes determining a dependency graph for a first MT. The dependency graph comprises a second MT from which the first MT depends. The first MT includes a query on one or more base tables and a lag duration value. The lag duration value indicates a maximum time period that a result of a prior refresh of the query can lag behind a current time instance. A tick period is selected for a set of ticks based on the lag duration value. The set of ticks corresponds to a set of aligned time instances. Refresh operations are scheduled for the first and second MTs at corresponding time instances from the set of aligned time instances. The corresponding time instances are separated by the tick period.Type: GrantFiled: September 13, 2022Date of Patent: September 12, 2023Assignee: Snowflake Inc.Inventors: Tyler Arthur Akidau, Tyler Jones, Daniel Mills, Leon Papke, Prasanna Rajaperumal, Daniel E. Sotolongo
-
Patent number: 11748327Abstract: A system or persistent table may be generated storing changelog information of a primary base table. The system table may then be used to create streams of relevant information. In some examples, the streams may read from the system table for information past a retention period of the primary table while reading from the primary table information in the retention period.Type: GrantFiled: January 21, 2022Date of Patent: September 5, 2023Assignee: Snowflake Inc.Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Daniel E. Sotolongo
-
Patent number: 11734301Abstract: Techniques described herein can enable stream replication. A first deployment can store a table including one or more streams. The techniques described herein can be used to replicate the table at a second deployment while replicating the one or more streams associated with the table. Select prior table versions and partitions in the table are copied to the second deployment to enable stream replication.Type: GrantFiled: March 23, 2022Date of Patent: August 22, 2023Assignee: Snowflake Inc.Inventors: Istvan Cseri, Tyler Jones, Daniel E. Sotolongo, Boyuan Zhang
-
Publication number: 20230237043Abstract: Techniques described herein can accelerate change data capture determinations such as stream reads, which show changes made to a table between two points in time. Three distinct row bitests that mark deleted, updated, inserted, rows in micro-partitions can be added as metadata for the table. These bitsets can be generated during DML operations and then stored as metadata of the new partition generated by the DML operations. The bitsets can then be used to generate streams showing the changes in the table between two points in time (changes interval).Type: ApplicationFiled: January 24, 2023Publication date: July 27, 2023Inventors: Tyler Arthur Akidau, Istvan Cseri, Fabian Hueske, Tyler Jones, Yevgeniy Kogan, Dzmitry Pauliukevich, Daniel E. Sotolongo
-
Patent number: 11620310Abstract: Techniques for triggering pipeline execution based on data change (transaction commit) are described. The pipelines can be used for data ingestion or other specified tasks. These tasks can be operational across account, organization, cloud region, and cloud provider boundaries. The tasks can be triggered by commit post-processing. Gates in the tasks can be set up to reference change data capture information. If the gate is satisfied, tasks can be executed to set up data pipelines.Type: GrantFiled: March 31, 2022Date of Patent: April 4, 2023Assignee: Snowflake Inc.Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Dinesh Chandrakant Kulkarni, Daniel Mills, Daniel E. Sotolongo, Di Fei Zhang
-
Publication number: 20230092367Abstract: Provided herein are systems and methods for a stream object configuration, including configuring and processing streams on views and streams on tables. For example, a method includes parsing, by at least one hardware processor, a received query to determine a plurality of data processing operations associated with a base table. A stream object on the base table is generated. The stream object includes a plurality of query results from executing the plurality of data processing operations and a corresponding plurality of time stamps of times the plurality of query results are obtained. A stream expansion of the stream object is performed. The stream expansion is based on computing changes to the base table from the plurality of data processing operations.Type: ApplicationFiled: October 24, 2022Publication date: March 23, 2023Inventors: Istvan Cseri, Tyler Jones, Daniel Mills, Daniel E. Sotolongo
-
Publication number: 20230070152Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.Type: ApplicationFiled: October 25, 2022Publication date: March 9, 2023Inventors: Tyler Arthur Akidau, Istvan Cseri, Tyler Jones, Daniel E. Sotolongo, Zhuo Zhang
-
Patent number: 11593345Abstract: Techniques described herein can accelerate change data capture determinations such as stream reads, which show changes made to a table between two points in time. Three distinct row bitests that mark deleted, updated, inserted, rows in micro-partitions can be added as metadata for the table. These bitsets can be generated during DML operations and then stored as metadata of the new partition generated by the DML operations. The bitsets can then be used to generate streams showing the changes in the table between two points in time (changes interval).Type: GrantFiled: January 21, 2022Date of Patent: February 28, 2023Assignee: Snowflake Inc.Inventors: Tyler Arthur Akidau, Istvan Cseri, Fabian Hueske, Tyler Jones, Yevgeniy Kogan, Dzmitry Pauliukevich, Daniel E. Sotolongo