Abstract: Implementing a database system using a plurality of sequentially ordered drives to store sequential portions of columns of the database, but where the database system is usable by a system configured for use with in-memory database systems. The method includes loading one or more columns of a database into main memory of a computing system as a table based database. The method further includes dividing a column of the database into sequential portions. Each of the sequential portions is of sufficient size to create efficient transfers using hard disk controllers to transfer an entire sequential portion. The method further includes writing each of the sequential portions sequentially onto sequentially ordered drives in a round robin fashion such that sequential portions of the column are on sequential drives.

Abstract: An index to be backed up in a computing environment is identified. A staged copy of the index to be backed up is created. The staged copy of the index is segmented. The segments are uploaded to a pre-selected location.

Abstract: A high-performance gridded database protocol for storing, arranging, and extracting gridded data includes associating values for a single grid cell and storing them together to extract as many useful values as possible from a single read operation. Gridded data is stored in a geographically-indexed cylindrical grid that permits efficient data extraction for a particular location while maximizing efficiency of read operations. Cylinders of values are built by grouping grids that are related to each other so that when data for a location is to be extracted, a minimal number of read operations is needed to retrieve an entire stack of data relevant to the location.

Abstract: A method for secure storage of secret data begins with an originating device transforming the secret data to produce a plurality of secret data shares and encrypting the plurality of secret data shares using unique encryption values of trusted agent modules of a dispersed storage network (DSN) to produce a plurality of encrypted secret data shares for storage in storage nodes of the DSN. Retrieval of the secret data begins with the originating device sending a secret data retrieval request to the trusted agent modules and recovering, by the trusted agent modules, the plurality of encrypted secret data shares from the storage nodes. The method continues with the trusted agent modules decrypting the plurality of encrypted secret data shares using a decryption function corresponding to the unique encryption values and sending the plurality of secret data shares to the originating device.

Abstract: A method for building a hash table over a subset of data in a data set includes mapping keys in the data set to values in the data set using multiple parallel computation threads. Each thread scans a subset of the keys and values and partitioning the subset of the keys and values into multiple partitions. A cumulative count for keys and values in each partition is determined. A hash table with space reserved for each partition is formed based on the determined cumulative counts. Each thread selects one or more partitions and inserts keys and values belonging to the selected one or more partitions into the hash table in the reserved space for those partitions.

Abstract: Methods and apparatus consistent with the invention provide the ability to organize, index, search, and present time series data based on searches. Time series data are sequences of time stamped records occurring in one or more usually continuous streams, representing some type of activity. In one embodiment, time series data is organized into discrete events with normalized time stamps and the events are indexed by time and keyword. A search is received and relevant event information is retrieved based in whole or in part on the time indexing mechanism, keyword indexing mechanism, or statistical indices calculated at the time of the search.

Abstract: Methods and apparatus consistent with the invention provide the ability to organize, index, search, and present time series data based on searches. Time series data are sequences of time stamped records occurring in one or more usually continuous streams, representing some type of activity. In one embodiment, time series data is organized into discrete events with normalized time stamps and the events are indexed by time and keyword. A search is received and relevant event information is retrieved based in whole or in part on the time indexing mechanism, keyword indexing mechanism, or statistical indices calculated at the time of the search.

Abstract: Methods and apparatus consistent with the invention provide the ability to organize, index, search, and present time series data based on searches. Time series data are sequences of time stamped records occurring in one or more usually continuous streams, representing some type of activity. In one embodiment, time series data is stored as discrete events time stamps. A search is received and relevant event information is retrieved based in whole or in part on the time stamp, a keyword indexing mechanism, or statistical indices calculated at the time of the search.

Abstract: Data entities in a client-server system are accessed. The client-server system comprises a set of clients, a server system, and a repository for storing a plurality of data entities. The server system comprises an inter-transactional cache, the inter-transactional cache being accessible for each client of the set of clients. A first client of the set of clients comprises a first intra-transactional cache. If a copy of a first data entity is in the inter-transactional cache, a version identifier of the original first data entity is read from the repository. If the copy of the first data entity is to be accessed with an exclusive lock, a copy of the copy of the first data entity is added to the first intra-transactional cache. The copy of the copy of the first data entity in the first intra-transactional cache is accessed for further processing of the first data entity by the first client.

Abstract: Embodiments include a method, system, and computer program product for acquiring a data repository, the data repository being associated with a log configured to receive metadata. Then, a content of the log with respect to at least one manipulation of the data repository is modified by adding first metadata of the metadata.

Abstract: The present disclosure relates to maintaining consistency in a content management system. A content management system generally includes a transactional database for storing content metadata and a non-transactional data repository for storing content. According to one embodiment, a content management system generates a set of content identifiers. The content management system receives a request to upload content to the content management system. Upon receiving the request, the content management system associates the content with a content identifier from the set of content identifiers and generates a name for the content from the content identifier. The content management system stores the content, with the generated name, in a non-transactional data repository and persists metadata related to the content in a transactional data store.

Abstract: In an approach for calculating a probability of a consistency violation of a transaction in a database management system, a processor receives a plurality of transactions within a predetermined time period. A processor identifies a first pattern in a first transaction of the plurality of transactions based on at least an isolation level of the transaction. A processor identifies a second pattern, wherein the second pattern is the cooperation between the first transaction and the second transaction. A processor determines that the first pattern of the first transaction substantially matches a preexisting pattern, wherein the preexisting pattern corresponds to values for a mathematical model for estimating a percentage of transactions in violation of consistency criteria. A processor extracts values from the first transaction based on the determined preexisting pattern. A processor calculates the mathematical model using the values from the first transaction and the values from the preexisting pattern.

Abstract: Extract, transform, and load application (ETL) complexity management framework systems and methods are described herein. The present disclosure describes systems and methods that reduce the complexity in managing ETL flow and correcting errant data that is subsequently identified. One or more methods include defining an ETL job definition, defining a data asset definition, defining a data asset dependency definition, receiving an ETL flow to provide execution of one or more ETL flow steps, providing retrieval of data from a source data asset, applying a data control to the source asset data, and producing an ETL job registration, a data asset status, a latest asset available date, a data asset consumer identifier, and a target data asset based on at least one of the ETL job definition, the data asset definition, the data dependency definition, and the source asset data.

Abstract: A method and associated system for generating a service specification of a solution. Each process of at least one process is decomposed into at least one atomic service process. The service specification of the solution to be compatible with both a process model framework that includes the at least one process and a data model framework that includes at least one data element that is indirectly matched to the at least one process. The service specification represents a respective process interface of each atomic service process that performs a respective atomic service in the solution. The customized service specification is stored in a database.

Abstract: A method is provided to verify the computational results of a transaction processing system utilizing cloud resources in a high-availability and scalable fashion. A transaction is allowed to modify an application's state only if the validity of the result of the processing of the transaction is verified across the majority of the participating child nodes in the cloud. Otherwise, the transaction is aborted.

Abstract: Transactional data accesses are performed in a data storage system, where the data storage system is configured to store a plurality of data objects identified by respective key values. A request is received to modify the value of a particular data object. The request specifies a modified object value and a key value identifying the data object to be modified. In response to the request, a transactional data object is created, the transactional data object specifying a transaction identifier, the original object value and the modified object value. The transactional data object is stored in the data storage system, and associated with the key value.

Abstract: A method for upgrading a database schema in real time, wherein the database schema is associated with a plurality of databases is provided. The method may include installing a first plugin on a database upgrade process associated with a database. The method may include installing a second plugin on each database instance within the plurality of databases. The method may include determining a delta associated with a final target version of the database schema. Additionally, the method may include outlining an existence of an incongruence, an existence of an override or an existence of a customization removal based on the determined delta. The method may include performing a peer-to-peer broadcasting analysis between each database. The method may include defining a migration strategy for each database instance within the plurality of databases based on the peer-to-peer broadcasting analysis. The method may further include creating a script for upgrading the database schema.

Abstract: A first request may be received to update a first set of values. The first set of values may be stored at a first location within a first data page of a database. The first location may be read-only. In response to the receiving of the first request, a first set of records may be inserted into a second data page. The first set of records may include the update of the first set of values. In response to the inserting, a forward pointer may be stored in the first data page that points to the first set of records on the second data page. One or more committed values may be identified on the second data page. In response to the identifying, the one or more committed values may be merged from the second data page to a third data page.

Abstract: An information retrieval method, process, and apparatus are provided which includes iterative or parametric data set querying. The result of each query iteration is displayed in an easy to analyze fashion, enabling the user to interactively refine the query with additional iterations. Each field of data in a data set is represented by a filter in a filter tree table. A user may graphically select and de-select filters using the filter tree table. The selections are converted into a filtering query that is run against the data set to produce filtered data. A summary query is then run against the results of the filtering query. The filtered data is displayed, along with the selected filters of the filter tree table. The filter tree table may also include and display other information related to each filter, such as an associated data item count as generated by the summary query. Further user input is accepted, with the user input further selecting or de-selecting data groupings to be displayed.

Abstract: There is provided an information processing apparatus including a device state detection unit configured to detect a device state of an electronic device, and a search condition setting unit configured to reference a database that stores information regarding the device state in association with a search condition used for a search for operation related information for the electronic device, and to extract the search condition corresponding to the device state.

Abstract: In one example in accordance with the present disclosure, a method for determining query plans includes obtaining a current query and generating a first literal and a first signature key from the current query. The method includes determining that the first signature key matches a second signature key stored in a parser cache and obtaining a literals data structure corresponding to the second signature key. The literals data structure comprises a group of literals, a list of high and low boundary intervals and a parameterization indicator. The method includes identifying, using the parameterization indicator, a parameterized literal in the group of literals. The method includes determining that a high and low boundary interval for the second parameterized literal contains the first literal and executing a stored query plan on the current query.

Abstract: In various embodiments, methods and systems for cross-filtering based on configurable direct relationships are provided. A selection of a first endpoint of a first table and a second endpoint of a second table is received. A direct relationship object is configured between the first table and the second table based on the first endpoint and the second endpoint. A cross-filtering direction and endpoint cardinality settings can also be configured for the direct relationship object. The direct relationship object may be configured as a many-to-one, one-to-many, or one-to-one. Automatic cross-filtering is configurable and can be set to cross-filter in one direction, the reverse direction, or both directions, with bidirectional cross filtering being a default configuration. The direct relationship object between the first endpoint and the second is configurable, such that, cross-filtering the first table and the second table to select rows for cross-filtering results is based on the cross-filtering direction.

Abstract: A dependency is created between a first search query and a second search query. The first search query defines a first data processing pipeline and the second search query defines a second data processing pipeline that extends the first data processing pipeline. A modification is detected to the first data processing pipeline defined by the first search query. Based on the modification to the first data processing pipeline being detected, the dependency is enforced such that the second data processing pipeline is modified to extend the modified first data processing pipeline. The modification to the first data processing pipeline can include a first set of pipelined commands corresponding to the first search query being modified, and the dependency can be enforced by causing a second set of pipelined commands corresponding to the second search query to be modified to include the modified first set of pipelined commands.

Abstract: A computer-implemented method for partitioning data for a query operation of one table of the database system is provided. The computer-implemented method comprises estimating a value distribution of the attribute in the result table based on a first value distribution of the attribute in the first column of the first table. The computer-implemented method further comprises determining boundaries for partitioning ranges of the attribute, based on the estimated value distribution, wherein the partitioning ranges correspond to a same number of rows of the result table. The computer-implemented method further comprises partitioning the first table with processing nodes of the query operation, based on the determined boundaries of partitioning ranges.

Abstract: A request is received to display at least a portion of a first events set and at least a portion of a second events set in an interleaved and visually distinct display format, where, in the interleaved and visually distinct display format, the at least a portion of the first events set is displayed in a visually distinct manner from the at least a portion of the second events set, and data from the at least a portion of the first events set is interleaved with data from the at least a portion of the second events set. In response to receiving the request, display is caused, on a user interface, of the at least a portion of the first events set and the at least a portion of the second events set in the interleaved and visually distinct display format.

Abstract: The present disclosure relates to systems and methods for generating a virtual data model of data from one or more source systems, where the exact number of each type of source system is not known. Templates for each source system type may be defined and then explicit views may be generated for each source system based on configuration data provided by a customer.

Abstract: A distributed database management system may comprise a plurality of computing nodes. A request to update an item maintained by the system may be acknowledged as durable and committed once an entry corresponding to the request has been written to a log file and quorum among the computing nodes has been achieved. Improved consistency may be achieved by maintaining snapshots of committed item states within queryable in-memory snapshot data structures. Range queries may be performed by merging a secondary index with the snapshots and applying filters. Projections may be completed by retrieving additional data from an item collection maintain on one or more storage devices.

Abstract: The disclosure generally describes computer-implemented methods, software, and systems for generating and executing queries. A query is received. A set of structure elements associated with the query is determined, wherein the structure elements are used for accessing, in a database, data tuples associated with the set of structure elements. Optionally, the set of structure elements is reduced for accessing data tuples to the structure elements selected. Contributing structure elements are added to the reduced set to not produce objects in the result set but for creating associated objects. Contributing structure elements are handled in an optimized way to keep the data set on which the query result set is calculated as small as possible. This is done by using the smallest set of characteristics in the group-by clause for retrieving the data set to calculate the requested set of structure elements. A result set of the query is provided.

Abstract: A system, computer-implemented method, and computer-program product embodiments for determining a cardinality estimate for a query. A cardinality estimator identifies a predicate in a query, where the predicate is split into a plurality of equivalence classes. The cardinality estimator then generates a plurality of equivalence graphs from the plurality of equivalence classes, one equivalence graph for an equivalence class. Spanning trees are identified from the plurality of equivalence graphs, and the cardinality estimator then determines the cardinality estimate for the query from the spanning trees.

Abstract: Certain example embodiments described herein relate to techniques for processing XML documents of potentially very large sizes. For instance, certain example embodiments parse a potentially large XML document, store the parsed data and some associated metadata in multiple independent blocks or partitions, and instantiate only the particular object model object requested by a program. By including logical references rather than physical memory addresses in such pre-parsed partitions, certain example embodiments make it possible to move the partitions through a caching storage hierarchy without necessarily having to adjust or encode memory references, thereby advantageously enabling dynamic usage of the created partitions and making it possible to cache an arbitrarily large document while consuming a limited amount of program memory.

Abstract: A system and method for improving input/output and memory utilization in a database are described. The system may implement an automatic vertical table decomposition process to achieve the improved input/output, storage and memory utilization in a database.

Abstract: A stream computing application may receive a stream of tuples to be processed by a plurality of stream operators. A first stream operator may receive a data stream, and the first stream operator may be configured to accumulate the data stream according to a window before performing one or more operations and transmitting tuples to a second stream operator. The second stream operator may request that the first stream operator send out tuples before the first stream operator's window has terminated. In response, the first stream operator may perform its operations on the tuples that it had accumulated before the request came in and then transmit a group of tuples with the intermediate results to the second stream operator.

Abstract: Systems, devices, and methods for contextual management extract context information and content information from a collection of stored digital content items; generate an index of the extracted context information and content information; store the indexed context information and content information; receive a query from a computing device, wherein the query is associated with a user; determine a context of the user; determine a context of a user activity; determine contents of the query; generate a recommendation in response to the query based on the context of the user, the context of the user activity, and the contents of the query; and return the recommendation to the computing device.

Abstract: A method and apparatus for selectively publishing user-provided content items to other users is provided. In some instances, the content items are excluded from being rendered to a user according to one or more blocks that block the user from viewing content items provided by certain members. Managing blocks and excluding content is more efficiently achieved through the implementation of a Bloom filter that contains the blocked pairs of users. The Bloom filter can, using few computing resources, be queried to determine if a block does not exist between two users. If the Bloom filter returns an indication that a block might exist, the apparatus determines if a block is stored in an LRU cache of recently retrieved blocks. If the LRU cache does not identify a block, then a call to retrieve one or more blocked pairs is made to a data storage.

Abstract: A method for sharing data based on user ranking is described. In one embodiment, the method includes determining a location of a computing device, identifying a request to share data from the computing device with one or more other computing devices, and ranking potential recipients of the data from the computing device based on at least one of location data, network communication data, and contact information.

Abstract: One or more systems and/or techniques are provided for automatic closed captioning for media content. In an example, real-time content, occurring within a threshold timespan of a broadcast of media content (e.g., social network posts occurring during and an hour before a live broadcast of an interview), may be accessed. A list of named entities, occurring within the social network data, may be generated (e.g., Interviewer Jon, Interviewee Kathy, Husband Dave, Son Jack, etc.). A ranked list of named entities may be created based upon trending named entities within the list of named entities (e.g., a named entity may be ranked higher based upon a more frequent occurrence within the social network posts). A dynamic grammar (e.g., library, etc.) may be built based upon the ranked list of named entities. Speech recognition may be performed upon the broadcast of media content utilizing the dynamic grammar to create closed caption text.

Abstract: In one embodiment, a web browser-based scheme for combining structured data, infographic definitions, and visual styling information to render infographics and aggregate collections of infographics, referred to herein as “Vizumes” and “Personas.” In exemplary embodiments of the disclosure, a relational database and/or a file system stores user data, infographic definitions, templates and palettes; combines these elements to produce individual infographic representations or a collection of infographic/visualizations (Vizumes) on a single canvas; allows users to choose different infographic visualizations of the same underlying data; and allows users to change the layout, font style, and color palette to instantly produce different visual presentations from the same data.

Abstract: A system that enables a user to configure alert actions based on search results generated by a query is disclosed. During operation, the system presents an alert user interface (UI) to a user, wherein the alert UI enables the user to configure one or more alert actions to be performed based on the search results. Next, the system receives alert configuration information from the user through the alert UI, wherein the alert configuration information includes tokens representing parameters associated with the query and the search results. Then, while generating an alert associated with the search results, the system performs a token substitution operation that substitutes tokens in the alert configuration information with corresponding parameters from the search results to generate a payload that is communicated to alert-generating functionality. This token substitution allows the parameters to be used by the alert-generating functionality while performing the one or more alert actions.

Abstract: A computing device receives a trigger to surface relevant content. The device also obtains a variety of different types of cross-source contextual information. Items of content are identified and relevancy weights are obtained based on the contextual information. A relevancy is calculated, based on the relevancy weights, for each item of content. The items of content are surfaced.

Abstract: An event limited field picker for a search user interface is described. In one or more implementations, a service may operate to collect and store data as events each of which includes a portion of the data correlated with a point in time. Clients may use a search user interface perform searches by input of search criteria. Responsive to receiving search criteria, the service may operate to apply a late binding schema to extract events that match the search criteria and provide search results for display via the search user interface. The search user interface exposes an event limited field picker operable to make selections of fields with respect to individual events in a view of the search results. In response to receiving an indication of a fields selected via the picker, visibility of selected fields may be updated to control which field and values are included in different views.

Abstract: Certain embodiments facilitate developing an application using a client-side object model to data source mapping. This may involve defining data use on the client side of the application in various ways that simplify the development necessary to implement data access features. In certain embodiments, data access and synchronization between a client and a data source is accomplished by configuring the client to map to a set of remote procedure call operations for a given data source. For example, the user may implement or confirm update, create, delete, and retrieve function calls for a given data source such that the application can implement data access and updating using those functions. Embodiments use persistent object identifiers, change logs, and/or data bindings in a client to provide various features and benefits. Certain embodiments also facilitate conflict resolution, demand paging, and lazy loading.

Abstract: Example embodiments of the present invention relate to a method, a system, and a computer program product for federated service requests of a logical data store comprising a plurality of heterogeneous data stores. The method includes providing a data abstraction services layer as a control plane for federated service requests of a logical data store, comprising a plurality of heterogeneous data stores, by an application operating on a processing device, receiving a service request at the data abstraction services layer from the application operating on the processing device directed to the logical data store, and returning results of the service request from the data abstraction services layer to the application operating on the processing device.

Abstract: A field extraction template simplifies the creation of field extraction rules by providing a user with a set of field names commonly assigned to a certain type of data, as well as guidance on how to extract values for those fields. These field extraction rules, in turn, facilitate access to certain “chunks” of the data, or to information derived from those chunks, through named fields. A field extraction template comprises at least a set of field names and ordering data for the field names. The ordering data indicates index positions that are associated with at least some of the field names. A delimiter is specified for splitting data items into arrays of chunks. The chunk of a data item that belongs to a given field name is the chunk whose position within the item's array of chunks is equivalent to the index position associated with the given field name.

Abstract: According to one embodiment, a method of copying a dataset associated with a first extract, transform, and load (ETL) job in a first data integration system to a second data integration system comprises copying executable code associated with the first ETL job from the first to the second system. Operating system software, integration system software, and file system structure are substantially identical between the first and second systems. The method further comprises executing the second ETL job to read the dataset from the first data integration system and write the dataset to the second data integration system. The second ETL job is associated with configuration parameters specifying storage resources in the first system associated with the dataset and destination parameters specifying storage resources in the second system. The method further comprises copying metadata generated by the second ETL job from the first to the second data integration system.

Abstract: Approaches for providing a customizable external data feed to a recipient. Metadata that identifies a plurality of configuration parameters that define characteristics of the external data feed are stored. A plurality of executable jobs using the configuration parameters defined in the metadata are programmatically generated. At least a first portion of executable jobs are executed to extract a data set identified by the configuration parameters from a persistent data store. At least a second portion of executable jobs are executed to format and transform the data set according to the configuration parameters. The external data feed, which comprises the formatted and transformed data set, is provided to the recipient.

Abstract: A method and an apparatus of maintaining data for online analytical processing in a database system. The method includes: tracking a changed page in a main process; and synchronizing the changed page to a child process for online analytical processing. In the method and apparatus of maintaining data for online analytical processing, the changed pages are tracked and then the child process is synchronized with the changed pages. Therefore, periodic forking is avoided, fork overhead due to periodic forking in the prior art is removed, the synchronization is faster since only the changed pages are synchronized, and the performance of online data processing is enhanced.

Abstract: The invention provides for checking data quality of data of an application program by a data quality management system. At least one of a plurality of jobs are executed for evaluating the data for compliance with one or more quality criteria. The runtime behavior of the at least one executed job is monitored to determine a current runtime behavior of the executed job. The monitored job is reclassified by reallocating the job to a job set representing the determined current runtime behavior.

Abstract: A processing platform integrates ETL (extract, transform, and load), real time stream processing, and “big data” data stores into a high performance analytic system that runs in a public or private cloud. The platform performs real time pre-storage enrichment of data records to form a single comprehensive record usable for analytics, searching and alerting. The platform further supports sharing of components and plug-ins and performs automatic scaling of resources based on real time resource monitoring and analysis.

Abstract: Systems and methods are provided for facilitating the transformation of data from a tabular data set organized according to a data schema to an object based data set organized according to a data ontology. The provided systems and methods offer a graphical user interface for mapping the tabular based data to the object based data set according to the data ontology. The tabular based data may be transformed according to the mapping.

Abstract: A column set server adapts to data use patterns by data consumers by modifying how a table produced by a data producer is partitioned into separate column sets to reduce the waste incurred in accessing the data by multiple consumers of the data. For example, the column set server adjusts a column set distribution when a new consumer process is added, when one is retired, or when relative data set size ratios change.