Abstract: A system and method of improved performant content source crawling. A content source crawling technology that is performant and focused on an initial rapid crawling of a content source to find specific file signatures so that it can determine an inventory of files that have been modified since the last full index event, thereby minimizing the time and computing resources necessary to perform a full crawl on just the select files to update to the search index. A method of Signature Flagging is disclosed and is used to selectively crawl the metadata about contained files and folders found within a content source in order to create that information in an Index within the Shinydocs Search Library.

Abstract: A system and method for extracting full text from source files of any size in such a manner that all text can be rapidly and accurately searched via search engine while maintaining satisfactory system performance. A software tool that integrates with file shares or an enterprise content management system with the ability to merge and serialize the results of the extracted text via surrogate index files. The software tool is configured to handle text from large files in order to extract, store and access the file such that all of the text is searchable, regardless of the size of the original document in a reasonable amount of time. The Analytics Engine will omit this field during search and handle this field separately from other fields.