Abstract: A system for knowledge graph construction. The system includes a computing device. The computing device has a processor and a storage device storing computer executable code. The computer executable code, when executed at the processor, is configured to: define entities and relations of the knowledge graph; provide documents having sentences; convert the sentences into fix length sentence embeddings and regard the sentence embeddings as primary capsule layers; use a set transformer to learn entity capsules and relation capsules from the primary capsule layers; for each triple, project head and tail entities from entity space to the specific relation space, and determine the relation exists when the sum of the projected head entity vector and the relation vector substantially equals to the projected tail entity vector; and construct the knowledge graph using the head entity, the tail entity, and the determined relation.

Abstract: A system and a method for object detection using augmented training dataset. The system includes a computing device, which is configured to: provide a two-dimensional (2D) image, extract feature vectors and estimate depths of 2D image pixels, generate a point cloud using the feature vectors and the depths, and project the point cloud using a new camera pose to obtain a projected image. The 2D image has a bounding box enclosing an object and labeled with the object. Each pixel within the bounding box is named bounding box pixel, each point in the point cloud corresponding to the bounding box pixel is named bounding box point, each image pixel in the projected image corresponding to the bounding box point is named projected bounding box pixel, and a projected bounding box is defined using the projected bounding box pixels and labeled with the object.

Abstract: An experience-aware anomaly processing system and a method for an experience-aware anomaly processing system are provided. The experience-aware anomaly processing system comprises an anomaly detection module configured to receive geographic location data with corresponding time information of a target object, and analyze target object behavior based on the geographic location data with corresponding time information of the target object; a user feedback module configured to receive user feedback from a user and model user feedback behavior when the user receives an alarm message indicating the target object is abnormal; and a decision module configured to receive user setting from the user, and make a detection decision through fusing target object behavior information corresponding to the target object behavior, user feedback behavior information corresponding to the user feedback behavior, and the user setting.

Abstract: A method, a device, and a non-transitory computer readable medium for item recommendation based on visual elements. The method includes: determining, by one or more processors, visual elements from an item image of an item; generating, by the one or more processors, an element descriptor for the item based on at least a part of the visual elements; and calculating, by the one or more processors, a compatibility value between the element descriptor and one or more other element descriptors for one or more other items.

Abstract: A system and a method for cooperative maneuvering and cooperative risk warning of vehicles. The system includes monocular surveillance cameras, local computing devices, and a master server. Each local computing device receives video frames from the camera, detects and tracks vehicles from the video frames, and converts the video frames to bird-view. Each detected vehicle is represented by a detection vector having first dimensions representing two dimensional (2D) parameters of the vehicle and second dimensions representing three dimensional (3D) parameters of the vehicle. Tracking of the vehicles is performed by minimizing loss calculated based on the first dimensions and the second dimensions of the detection vectors. The master server receives bird-views from different computing devices and combines the bird-views into a global bird-view, and performs cooperative maneuvering and cooperative risk warning of the vehicles using the global bird-view.

Abstract: A system for knowledge graph construction. The system includes a computing device. The computing device has a processor and a storage device storing computer executable code. The computer executable code, when executed at the processor, is configured to: define entities and relations of the knowledge graph; provide documents having sentences; convert the sentences into fix length sentence embeddings and regard the sentence embeddings as primary capsule layers; use a set transformer to learn entity capsules and relation capsules from the primary capsule layers; for each triple, project head and tail entities from entity space to the specific relation space, and determine the relation exists when the sum of the projected head entity vector and the relation vector substantially equals to the projected tail entity vector; and construct the knowledge graph using the head entity, the tail entity, and the determined relation.

Abstract: A system and a method for pose tracking, particularly for top-down, online, multi-person pose tracking. The system includes a computing device having a processor and a storage device storing computer executable code. The computer executable code, when executed at the processor, is configured to: provide a plurality of sequential frames of a video, the sequential frames comprising at least one keyframe and a plurality of non-keyframes; for each of the non-keyframes: receive a previous inference bounding box of an object inferred from a previous frame; estimate keypoints from the non-keyframe in an area defined by the previous inference bounding box to obtain estimated keypoints; determine object state based on the estimated keypoints, wherein the object state comprise a “tracked” state and a “lost” state; and when the object state is “tracked,” infer an inference bounding box based on the estimated keypoints to process a frame next to the non-keyframe.

Abstract: A method, a device and a non-transitory computer readable medium for tracking poses of multiple objects. The method includes detecting, by a processor, objects from each of a plurality of consecutive frames in a video sequence; estimating, by the processor, a pose of each of the objects within each of the plurality of consecutive frames; and tracking, by the processor, the poses of each of the objects across the plurality of consecutive frames.

Abstract: A system and a method for cooperative maneuvering and cooperative risk warning of vehicles. The system includes monocular surveillance cameras, local computing devices, and a master server. Each local computing device receives video frames from the camera, detects and tracks vehicles from the video frames, and converts the video frames to bird-view. Each detected vehicle is represented by a detection vector having first dimensions representing two dimensional (2D) parameters of the vehicle and second dimensions representing three dimensional (3D) parameters of the vehicle. Tracking of the vehicles is performed by minimizing loss calculated based on the first dimensions and the second dimensions of the detection vectors. The master server receives bird-views from different computing devices and combines the bird-views into a global bird-view, and performs cooperative maneuvering and cooperative risk warning of the vehicles using the global bird-view.

Abstract: A system and a method for recommending a set of fashion items. The method includes: providing fashion categories, fashion graphs each corresponding to one fashion category, and fitting formulas for each fashion graph; receiving a query fashion item having a query fashion attribute; selecting a fashion category, where the query fashion attribute in the selected fashion category has a highest number of fashion items; performing a similarity search between image of the query fashion item and images of fashion items in the fitting formulas; mapping the searched fitting formula to the fashion graph of the selected fashion category to obtain nodes matching the fashion items in the searched fitting formula; and recommending the set of fashion items respectively selected from fashion items in the matched nodes. The fitting formulas are generated based on co-appearance of fashion items from any two nodes of the corresponding fashion graph.

Abstract: A system and a method for pose tracking, particularly for top-down, online, multi-person pose tracking. The system includes a computing device having a processor and a storage device storing computer executable code. The computer executable code, when executed at the processor, is configured to: provide a plurality of sequential frames of a video, the sequential frames comprising at least one keyframe and a plurality of non-keyframes; for each of the non-keyframes: receive a previous inference bounding box of an object inferred from a previous frame; estimate keypoints from the non-keyframe in an area defined by the previous inference bounding box to obtain estimated keypoints; determine object state based on the estimated keypoints, wherein the object state comprise a “tracked” state and a “lost” state; and when the object state is “tracked,” infer an inference bounding box based on the estimated keypoints to process a frame next to the non-keyframe.

Abstract: A method, a device, and a non-transitory computer readable medium for item recommendation based on visual elements. The method includes: determining, by one or more processors, visual elements from an item image of an item; generating, by the one or more processors, an element descriptor for the item based on at least a part of the visual elements; and calculating, by the one or more processors, a compatibility value between the element descriptor and one or more other element descriptors for one or more other items.

Abstract: A method for optimized wake-up strategy via sleeping stage prediction with recurrent neural networks is provided. The method comprises receiving a set of raw sensory data of a user's sleeping status; translating the set of raw sensory data into a plurality of sleeping stages; saving the plurality of sleeping stages as a history of sleeping stages; determining a current sleeping stage; predicting a plurality of future sleeping stages of the user in an upcoming interval; calculating a wakeup score of the current sleeping stage and a wakeup score of each future sleeping stage; determining an optimized sleeping stage for wake up based on the wakeup score through comparing the wakeup score of the current sleeping stage and the wakeup scores of the future sleeping stages; and triggering an alarm impulse to wake up the user at the optimized sleeping stage for wake up.

Abstract: A method, a device and a non-transitory computer readable medium for tracking poses of multiple objects. The method includes detecting, by a processor, objects from each of a plurality of consecutive frames in a video sequence; estimating, by the processor, a pose of each of the objects within each of the plurality of consecutive frames; and tracking, by the processor, the poses of each of the objects across the plurality of consecutive frames.

Abstract: A method for optimized wake-up strategy via sleeping stage prediction with recurrent neural networks is provided. The method comprises receiving a set of raw sensory data of a user's sleeping status; translating the set of raw sensory data into a plurality of sleeping stages; saving the plurality of sleeping stages as a history of sleeping stages; determining a current sleeping stage; predicting a plurality of future sleeping stages of the user in an upcoming interval; calculating a wakeup score of the current sleeping stage and a wakeup score of each future sleeping stage; determining an optimized sleeping stage for wake up based on the wakeup score through comparing the wakeup score of the current sleeping stage and the wakeup scores of the future sleeping stages; and triggering an alarm impulse to wake up the user at the optimized sleeping stage for wake up.

Abstract: An experience-aware anomaly processing system and a method for an experience-aware anomaly processing system are provided. The experience-aware anomaly processing system comprises an anomaly detection module configured to receive geographic location data with corresponding time information of a target object, and analyze target object behavior based on the geographic location data with corresponding time information of the target object; a user feedback module configured to receive user feedback from a user and model user feedback behavior when the user receives an alarm message indicating the target object is abnormal; and a decision module configured to receive user setting from the user, and make a detection decision through fusing target object behavior information corresponding to the target object behavior, user feedback behavior information corresponding to the user feedback behavior, and the user setting.

Abstract: A content retrieval method including: extracting a plurality of fingerprints including a plurality of video fingerprints and audio fingerprints from contents stored in a content database; determining representative video fingerprints of the video frames and representative audio fingerprints of the audio sequences; determining a data rate indicating a storage limitation and a coverage indicating a number of searching results to be returned; storing selected representative video fingerprints and representative audio fingerprints based on the storage limitation indicated by the data rate in a fingerprint database; receiving a query containing at least one of video data and audio data submitted by a user; extracting at least one query fingerprint representing the query; determining a number of fingerprints most matching the query fingerprint based on the coverage to generate search results indicating matching contents represented by the number of most matching fingerprints; and returning the search results to the

Abstract: A method for a mobile search-ready smart display system containing a smart display having a screen and at least one mobile device having a camera including extracting a plurality of video fingerprints from an ongoing content displayed on the screen. The ongoing content has video frames corresponding to the video fingerprints; determining one or more representative video fingerprints of the video frames from the plurality of video fingerprints; generating one or more encoded representative video fingerprints through encoding the one or more representative video fingerprints into a pixel translucency change; and delivering the one or more encoded representative video fingerprints via a screen-camera communication. The method also includes optimizing the encoding of the one or more representative video fingerprints and the delivering of the one or more encoded representative video fingerprints to find a balance between an error resilience and transmission time.

Abstract: In accordance with various embodiments of the disclosed subject matter, a method and a system for vision-centric deep-learning-based road situation analysis are provided. The method can include: receiving real-time traffic environment visual input from a camera; determining, using a ROLO engine, at least one initial region of interest from the real-time traffic environment visual input by using a CNN training method; verifying the at least one initial region of interest to determine if a detected object in the at least one initial region of interest is a candidate object to be tracked; using LSTMs to track the detected object based on the real-time traffic environment visual input, and predicting a future status of the detected object by using the CNN training method; and determining if a warning signal is to be presented to a driver of a vehicle based on the predicted future status of the detected object.

Abstract: A method for a mobile search-ready smart display system containing a smart display having a screen and at least one mobile device having a camera including extracting a plurality of video fingerprints from an ongoing content displayed on the screen. The ongoing content has video frames corresponding to the video fingerprints; determining one or more representative video fingerprints of the video frames from the plurality of video fingerprints; generating one or more encoded representative video fingerprints through encoding the one or more representative video fingerprints into a pixel translucency change; and delivering the one or more encoded representative video fingerprints via a screen-camera communication. The method also includes optimizing the encoding of the one or more representative video fingerprints and the delivering of the one or more encoded representative video fingerprints to find a balance between an error resilience and transmission time.