Patents by Inventor Nileema Shingte
Nileema Shingte has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11960505Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.Type: GrantFiled: May 19, 2022Date of Patent: April 16, 2024Assignee: Snowflake Inc.Inventors: Vasile Paraschiv, Saurin Shah, Marianne Shaw, Nileema Shingte
-
Publication number: 20240111762Abstract: Disclosed herein are systems and methods for efficiently querying external tables. In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.Type: ApplicationFiled: December 1, 2023Publication date: April 4, 2024Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11899646Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.Type: GrantFiled: March 30, 2023Date of Patent: February 13, 2024Assignee: Snowflake Inc.Inventors: Selcuk Aya, Thierry Cruanes, Istvan Cseri, Benoit Dageville, Marcia Feitel, Steven P. Herbert, Xinglian Liu, James Malone, Subramanian Muralidhar, Muthunagappan Muthuraman, Polita Paulus, Marianne Shaw, Nileema Shingte, Wai Sing Wong, Jiaqi Yan
-
Publication number: 20230401193Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.Type: ApplicationFiled: March 30, 2023Publication date: December 14, 2023Inventors: Selcuk Aya, Thierry Cruanes, Istvan Cseri, Benoit Dageville, Marcia Feitel, Steven P. Herbert, Xinglian Liu, James Malone, Subramanian Muralidhar, Muthunagappan Muthuraman, Polita Paulus, Marianne Shaw, Nileema Shingte, Wai Sing Wong, Jiaqi Yan
-
Patent number: 11841849Abstract: In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.Type: GrantFiled: July 15, 2022Date of Patent: December 12, 2023Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11675780Abstract: Disclosed herein are embodiments of systems and methods for partition-based scanning of external tables for query processing. In an example embodiment, a database platform receives a query that includes one or more predicates, where the query is directed at least to data in an external table that is stored in an external storage platform that is external to the database platform. The database platform identifies, based on metadata that summarizes the data in the external table, one or more partitions of the external table that potentially include data that satisfies the one or more predicates. The database platform also identifies, from the one or more identified partitions, data that satisfies the one or more predicates. The database platform sends a response to the query to the client, the response comprising the data satisfying the one or more predicates.Type: GrantFiled: February 9, 2022Date of Patent: June 13, 2023Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11507571Abstract: Systems, methods, and devices for generating a materialized view over an external table. A method includes connecting a database platform to an external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating, by the database platform, a materialized view over the external table. The method includes receiving a notification that a modification has been made to the external table, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table. The method includes, in response to the external table being modified, refreshing the materialized view such that the materialized view comprises an accurate representation of the external table.Type: GrantFiled: April 16, 2019Date of Patent: November 22, 2022Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Publication number: 20220269674Abstract: Disclosed herein are embodiments of systems and methods for partition-based scanning of external tables for query processing. In an example embodiment, a database platform receives a query that includes one or more predicates, where the query is directed at least to data in an external table that is stored in an external storage platform that is external to the database platform. The database platform identifies, based on metadata that summarizes the data in the external table, one or more partitions of the external table that potentially include data that satisfies the one or more predicates. The database platform also identifies, from the one or more identified partitions, data that satisfies the one or more predicates. The database platform sends a response to the query to the client, the response comprising the data satisfying the one or more predicates.Type: ApplicationFiled: February 9, 2022Publication date: August 25, 2022Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11397729Abstract: Disclosed herein are systems and methods for pruning external data. In an embodiment, a database platform receives a query directed at least in part to external data in an external table on an external data storage platform. The external table includes partitions that correspond to storage locations in a source directory of the external data storage platform. The storage locations contain files that contain the external data. The database platform identifies, from external-table metadata that is stored by the database platform and that maps the partitions of the external table to the storage locations in the source directory, a subset of the partitions as including data that potentially satisfies the query. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.Type: GrantFiled: November 19, 2021Date of Patent: July 26, 2022Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11354331Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.Type: GrantFiled: October 30, 2020Date of Patent: June 7, 2022Assignee: Snowflake Inc.Inventors: Vasile Paraschiv, Saurin Shah, Marianne Shaw, Nileema Shingte
-
Patent number: 11354316Abstract: Disclosed herein are systems and methods for selective scanning of external partitions. In an embodiment, a database platform receives a query directed at least in part to an external table stored on an external data storage platform. The external table is partitioned into partitions corresponding to storage locations in the external data storage platform. The database platform prunes, using external-table metadata that is stored by the database platform and that maps the partitions of the external table to the storage locations in the external data storage platform, those partitions that do not potentially contain data that satisfies the query. The database platform identifies data that satisfies the query by scanning any one or more of the partitions of the external table that were not pruned, and responds to the query at least in part with the identified data that satisfies the query.Type: GrantFiled: December 23, 2021Date of Patent: June 7, 2022Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11347728Abstract: The subject technology receives a notification that a modification has been made to an external table, the modification comprising inserting at least one row of new data to the external table, the at least one row corresponding to a first micro-partition that includes a first portion of data from the external table prior to the inserting. The subject technology, in response to the notification indicating the modification to the external table, generates a new micro-partition different from the first micro-partition, the new micro-partition including the inserted at least one row of new data and the first portion of data from the external table. The subject technology generates a refreshed materialized view based at least in part on the generated new micro-partition such that the refreshed materialized view comprises a representation of the external table after the modification has been made.Type: GrantFiled: August 31, 2021Date of Patent: May 31, 2022Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11347775Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.Type: GrantFiled: August 31, 2021Date of Patent: May 31, 2022Assignee: Snowflake Inc.Inventors: Vasile Paraschiv, Saurin Shah, Marianne Shaw, Nileema Shingte
-
Publication number: 20220121673Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.Type: ApplicationFiled: October 30, 2020Publication date: April 21, 2022Inventors: Vasile Paraschiv, Saurin Shah, Marianne Shaw, Nileema Shingte
-
Publication number: 20220121683Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.Type: ApplicationFiled: August 31, 2021Publication date: April 21, 2022Inventors: Vasile Paraschiv, Saurin Shah, Marianne Shaw, Nileema Shingte
-
Publication number: 20220114180Abstract: Disclosed herein are systems and methods for selective scanning of external partitions. In an embodiment, a database platform receives a query directed at least in part to an external table stored on an external data storage platform. The external table is partitioned into partitions corresponding to storage locations in the external data storage platform. The database platform prunes, using external-table metadata that is stored by the database platform and that maps the partitions of the external table to the storage locations in the external data storage platform, those partitions that do not potentially contain data that satisfies the query. The database platform identifies data that satisfies the query by scanning any one or more of the partitions of the external table that were not pruned, and responds to the query at least in part with the identified data that satisfies the query.Type: ApplicationFiled: December 23, 2021Publication date: April 14, 2022Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Publication number: 20220075776Abstract: Disclosed herein are systems and methods for pruning external data. In an embodiment, a database platform receives a query directed at least in part to external data in an external table on an external data storage platform. The external table includes partitions that correspond to storage locations in a source directory of the external data storage platform. The storage locations contain files that contain the external data. The database platform identifies, from external-table metadata that is stored by the database platform and that maps the partitions of the external table to the storage locations in the source directory, a subset of the partitions as including data that potentially satisfies the query. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.Type: ApplicationFiled: November 19, 2021Publication date: March 10, 2022Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11269869Abstract: Disclosed herein are systems and methods for processing queries over external tables. In an embodiment, a database platform receives a query directed at least to data in an external table stored in a storage platform that is external to the database platform. The database platform uses metadata that summarizes the data in the external table to identify one or more partitions of the external table as potentially including data satisfying the query, and generates a query plan that includes a plurality of discrete subtasks that collectively include instructions to scan the identified one or more partitions of the external table for data satisfying the query. The database platform assigns, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform, and refreshes the metadata in response to a threshold number of modifications being made to the external table.Type: GrantFiled: October 11, 2021Date of Patent: March 8, 2022Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Patent number: 11269868Abstract: Systems, methods, and devices for automated maintenance of external tables in database systems are disclosed. A method includes receiving, by a database platform, read access to content in an external data storage platform that is separate from the database platform. The method includes defining an external table based on the content in the external data storage platform. The method includes connecting the database platform to the external table such that the database platform has read access for the external table and does not have write access for the external table. The method includes generating metadata for the external table, the metadata comprising information about data stored in the external table. The method includes receiving a notification that a modification has been made to the content in the external data storage platform, the modification comprising one or more of an addition of a file, a deletion of a file, or an update to a file in a source location for the external table.Type: GrantFiled: March 31, 2021Date of Patent: March 8, 2022Assignee: Snowflake Inc.Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri
-
Publication number: 20220027368Abstract: Disclosed herein are systems and methods for processing queries over external tables. In an embodiment, a database platform receives a query directed at least to data in an external table stored in a storage platform that is external to the database platform. The database platform uses metadata that summarizes the data in the external table to identify one or more partitions of the external table as potentially including data satisfying the query, and generates a query plan that includes a plurality of discrete subtasks that collectively include instructions to scan the identified one or more partitions of the external table for data satisfying the query. The database platform assigns, based on the metadata, the plurality of discrete subtasks to one or more nodes in an execution platform, and refreshes the metadata in response to a threshold number of modifications being made to the external table.Type: ApplicationFiled: October 11, 2021Publication date: January 27, 2022Inventors: Subramanian Muralidhar, Benoit Dageville, Thierry Cruanes, Nileema Shingte, Saurin Shah, Torsten Grabs, Istvan Cseri