Video management with push to talk (PTT)
Embodiments include a system, method, and computer program product that enable video management of a plurality of video feeds with Push to Talk (PTT) communications. Some embodiments monitor a plurality of video feeds using a VMS graphics user interface (GUI) where a plurality of video feeds are video feed components of an interactive multimedia media object (IMMO) displayed in the VMS GUI. Some embodiments include receiving a selection of a first talk group that is associated with the IMMO, where members of the first talk group may view the video feed components and communicate via PTT. Subsequently, some embodiments include receiving a second selection of a second talk group, associating the second talk group with the IMMO, and enabling PTT communications among members of the second talk group and an operator of the VMS.
Latest Mutualink, Inc. Patents:
- Incident communications network with dynamic asset marshaling and a mobile interoperability workstation
- AMBIGUATED CONTROL FUNCTIONS FOR COVERT COMMUNICATIONS
- IMPLEMENTING PUSH-TO-TALK IN A MULTIMEDIA CONFERENCING SYSTEM
- Ambiguated control functions for covert communications
- SYSTEM FOR A PERSONAL WEARABLE MICRO-SERVER
This application is a continuation of U.S. application Ser. No. 15/725,953, filed Oct. 5, 2017, entitled Video Management Enabling Voice Communication Groups, which is a continuation of U.S. application Ser. No. 14/875,260, filed Oct. 5, 2015, entitled Video Management Defined Embedded Voice Communication Groups, which are incorporated herein by reference in their entireties.
BACKGROUND FieldThe embodiments generally relate to providing a video management software (VMS) system with push to talk (PTT) services in association with video management for members of a voice communication group.
BackgroundPresently, a plethora of disparate communications resources exist including resources using private wireless communications (e.g., public safety and first responder communications networks), public switched network communications resources, public wireless networks, networks of video surveillance devices, private security networks, and the like. For example, prior art video management software (VMS) systems typically enable a control operator to monitor one or more video streams provided by networked video surveillance cameras via a VMS graphical user interface (GUI). But in such systems the control operator may need to utilize multiple separate communication resources including, for example, a Push to Talk (PTT) voice communication system to communicate with field users and first responders information obtained from the monitored video streams of the VMS system.
A typical PTT voice communication system, such as a mobile radio system, enables users to exchange voice communications with each other within defined talk groups or designated radio channels. A PTT talk group is a logically defined voice communications group defined by a PTT user or PTT operator that allows users to send and receive voice messages from other member users in the talk group. The PTT voice communication system is often used by security, public safety and other private user groups to facilitate voice communications within a radio network's coverage area.
Conventional PTT systems also support half-duplex communication methods, e.g., a two-way radio, where only one member of a talk group can transmit and speak at a time, and the remaining members listen. When the transmit channel is released, other members may push to talk (e.g., key a radio) and seize control of the transmitting function until the transmit function is released. The transmit control function may be accomplished through control signaling which blocks other PPT users from acquiring control of the shared transmit function, or may be accomplished through other means such a voice activated detection (VAD) coupled with in-band or out of band control signaling which blocks transmission by other users while a voice communication is being transmitted.
Internet protocol systems using Voice over IP (VOIP) and other digital voice transmission over data network technologies enable client software operated on computing devices such as mobile smartphones to simulate PTT type functionality among a group of users. This IP based PTT technology enables users with a defined talk group to transmit and receive voice communications on a persistent basis subject to a user's PTT application client being actively run and being in active state such as being in the application foreground of an operating system. IP PTT may employ half duplex or full duplex communications, and may impose floor control using means similar to radio-based PTT systems. In either radio or IP based PTT systems, a central operator function serves to register and administer members of a talk group and a talk group is typically limited to enterprise use (members within the same agency or enterprise) or user-defined where an authorized user may establish a talk group and invite members via the central operator.
Disparate Push to Talk (PTT) Services
When a control operator detects or becomes aware of an event of interest, e.g., suspicious activity or an emergency situation, in a video feed that the control operator is monitoring, voice communication to field personnel is often required to respond to the object or event of interest. For example, the control operator may be in a security monitoring room operating a VMS application to monitor multiple surveillance cameras distributed across a parking lot. Upon observing within a video feed a potential crime, such as a carjacking attempt, the control operator may need to communicate with a security guard in physical proximity to the surveillance camera providing the video feed.
Typically, the control operator may further need to use a separate radio communication system to communicate with radio users, including the security guard, operating on the same channel or within the same talk group. Alternatively, the control operator may communicate via a telephone or mobile telephone call to a prescribed call number of the security guard. While operating disparate communication resources in this scenario may be possible, albeit inefficient, in the case where multiple events are observed concurrently across different video feeds, managing real-time communication with one or more talk groups in relation to the appropriate video feed becomes exceedingly cumbersome and unmanageable. Furthermore, even in the case of one observed event, the security guard may not be in proximity to the observed event and the control operator should have been communicating with another field personnel in closer proximity to reduce the response time in addressing the event of interest.
Inflexible Video Management Software (VMS) Systems
Traditional VMS systems provide monitoring of video cameras that are fixedly coupled to call box functions or intercom systems as in the case of video entry camera monitoring system. In this example, a visitor can initiate an audible notification which is sent to a control operator. The visitor can be seen by the operator, who can communicate with the observed visitor through an intercom box with a speaker and microphone. At most, recent adaptions of VMS systems use internet protocol (IP) networks to enable two-way voice communications between a video camera with speaker and microphone (whether embedded or separately coupled) and the operator using a VMS GUI control at a control station monitoring console.
In the current VMS systems noted above, the voice communication channel or system enables a one-to-one communications modality between the video monitoring camera and the communicating intercom device associated with the video camera. But the control operator observing an event of interest within a video feed often requires communication with multiple field personnel from one or more agencies. Moreover, the intercom device providing voice communications associated with the video feed is fixed either in being physically proximate to the video camera or physically coupled to or part of the video camera device. Therefore, field personnel must be physically proximate to the video camera to communicate with the control operator.
Furthermore, in an emergency situation where multiple events are observed concurrently across a plurality of video feeds, the operators may need to minimize the set up time required to communicate with talk group members proximate to one or more of the events. Additionally, the control operator may best address the emergency situation by directing field personnel and first responders to an event viewed from a video feed based on a field personnel's specific skill-set, expertise, and/or knowledge of how a specific type of equipment operates. Determining and identifying the requisite personnel having the necessary skills and/or equipment, however, can be time consuming and require the operator to relay multiple complex communication with and between various agencies.
In addition, disaster areas often have various levels of restricted or limited access areas for personnel, due to environmental hazards or for facilitating ease of movement of personnel and equipment or the performance of certain unimpeded operational tasks. Communicating restricted access information, including the description of geographic boundaries, communicating access credential levels, and interrogating the validity of access credentials with respect to various personnel can add another layer of complexity for the control operator in identifying the appropriate members of a talk group to communicate with in addressing an emergency situation observed across one or more video feeds of the VMS GUI.
Embodiments address, among other things, the problem that conventional VMS systems are disparate from PTT services or otherwise provide fixed and limited coupling to PTT services. Therefore, a control operator monitoring video feeds within a VMS GUI may need to separately identify appropriate PTT talk groups in relation to a specific video feed and independently operate a separate communication resource, e.g., a PTT communication system. In other examples, field members may need to be physically proximate to voice communication devices coupled to the video cameras on site. In addition, conventional PTT services do not consider geographic boundaries.
BRIEF SUMMARY OF THE INVENTIONWhat is needed is a system, method, and computer program product that embeds push to talk (PTT) functionality within a video management software (VMS) system such that disparate voice and video communication resources are integrated. In an embodiment, a VMS graphics user interface (GUI) provides the capability to monitor a plurality of video feeds provided by corresponding video cameras. Each of the video feeds may be a video feed component of a corresponding interactive multimedia object (IMMO) that is displayed in the VMS GUI. One or more video feeds displayed within the VMS GUI may be coupled to an embedded PTT control component. A defined talk group may be logically associated with the one or more video feeds by coupling the talk group to the embedded PTT control component. Upon detecting that the PTT control component is activated by an operator, the VMS system may enable PTT voice communications related to the one or more video feed views between the operator and each member comprising the defined talk group. In an embodiment, the talk group may be selected from a predefined talk group or a dynamically created group.
In an embodiment, the VMS system may determine a geolocation of each video camera providing a corresponding video feed of an IMMO. Based on determined geolocations, a plurality of IMMOs may be displayed on a map interface within the VMS GUI such that each IMMO is positioned on the map to correlate with the determined geolocation. The IMMO may include multiple components to integrate different communication resources including voice and video communications.
Further embodiments, features, and advantages, as well as the structure and operation of the various embodiments, are described in detail below with reference to accompanying drawings. It is noted that the embodiments are presented herein for illustrative purpose only. Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein.
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.
The drawing in which an element first appears is typically indicated by the leftmost digit or digits in the corresponding reference number. In the drawings, like reference numbers may indicate identical or functionally similar elements.
DETAILED DESCRIPTIONSystem
VMS system 102A-C may each include a variety of network interfaces (not shown) including wireless and wired network interfaces to support a variety of network protocols. Each VMS system 102, may implement and host one or more VMS applications for integrating PTT functionality across one or more PTT mobile devices 120 with the management of one or more video cameras 110 across one or more networks. In an embodiment, VMS system 102A, for example, may manage or monitor video cameras 110E-G through LAN 108 and video cameras 110A-D through Internet Protocol (IP) network 104.
Video cameras 110 may be mounted surveillance video cameras that are analog cameras or IP cameras, which may be powered over Ethernet cables (PoE). Such surveillance cameras may have pan, tilt, and zoom (PTZ) capabilities. In an embodiment, video camera 110 may capture a video feed and metadata associated with an operating status of the video camera to be transmitted, via a network, to one or more of VMS systems 102 managing video camera 110. The metadata may include at least one of the following information related to video camera 110: unique video camera identifier (ID), group video camera ID, one or more associated user IDs, geolocation information, timestamp of captured video, direction, inclination, angle, moment, velocity, acceleration, or other environmental parameters.
For example, video cameras 110A-D may be representative of surveillance cameras mounted near bus stops, at traffic intersections, or within office buildings. Video cameras 110A-B may be managed through VMS server 112 having connection to IP network 104. VMS server 112 may therefore forward video, audio, or metadata captured by video cameras 110A-B to VMS system 102A, for example, through IP network 104. In an embodiment, VMS server 112 may include digital video recorder (DVR) or network video recorder (NVR) capabilities in order to store or record video and metadata captured by video cameras 110A-B.
In an embodiment, video cameras, such as video cameras 110D, may be coupled to wireless access point (AP) 113 that transmits captured metadata to VMS system 102A through IP network 104. In a further embodiment, video cameras, such as video camera 110C, may include networking hardware to connect to IP network 104 directly. In an embodiment, one or more video cameras 110, such as video camera 110C, may be wearable video cameras worn by field personnel.
In an embodiment, to integrate various communications functionality, e.g., PTT functionality, within a VMS system, VMS system 102A may include VMS-PTT application server 116, database 115, network video recorder (NVR) 117, and interoperability gateway 118.
VMS-PTT application server 116 may host the VMS application or provide the functionality of the VMS application through one or more VMS graphic user interfaces (GUI) such as those exemplarily illustrated in
In an embodiment, VMS-PTT application server 116 may enable an operator of the VMS application to engage in PTT communications with members of a voice communications group (i.e., a talk group), such as field personnel operating one or more PTT mobile devices 120A-B of talk group 122A, that is logically associated with one or more selected video feeds of the one or more video cameras 110. In an embodiment, when the operator enables an active transmission mode, the operator's PTT client transmits the operator's voice communication to each member of the talk group, specifically, to each PTT client of respective members' PTT mobile devices. PTT clients may also enable transmission and receipt of real time voice communications, video streams, files, text, and other data resources between and among users who are members of a talk group.
For example, the operator may detect a car crash being captured by video camera 110D that is logically associated with talk group 122A having members that are within, for example, a three mile radius of video camera 110D. Thus, the operator may talk to field personnel using PTT mobile devices 120A-B about the incident captured by video camera 110D. The operator may access the VMS application through LAN network 108 using monitoring device 124A or, in an embodiment, through IP network 104 using monitoring device 124B. In an embodiment, the operator role may be permitted to a field user operating one or more of PTT mobile devices 120.
Talk groups 122, such as talk group 122A, is a logically defined communications group comprised of PTT users, such as field personnel using PTT mobile devices 120. In an embodiment, a PTT mobile device, such as PTT mobile device 120A, may be a member of more than one talk group including, for example, talk groups 122A-B based on determinations by one or more VMS systems 102. Similarly, an operator may be capable of communicating with one or more talk groups 122 established by VMS system 102A. In an embodiment, VMS system 102A may use parameters and characteristics of field personnel, their corresponding PTT mobile devices 120, operator settings, video cameras 110, or any combination thereof to determine that talk group 122B, for example, includes PTT mobile devices 120A-C. In an embodiment, upon determining talk group members, VMS system 102A may initiate a process to invite and add the field member as a member of the talk group.
Interoperability gateway 118 coupled to VMS-PTT application server 116 may be further coupled to two-way mobile radio systems such as land mobile radio (not shown) serving mobile radios and other end point communications devices (“off-net PTT devices”) that are members of other off-net PTT networks (not shown) such as the public telephone switched networks, other land mobile radio networks, mobile telephone networks, and private multimedia broadband networks. Through this coupling, interoperability gateway 118 may enable IP voice and data messages to be transmitted to and received from the off-net PTT devices not connected to IP network 104.
In an embodiment, interoperability gateway 118 may include logic to enable multimedia communications including, but not limited to, video file or video camera feed data to be transmitted to or shared with a separate or distinct VMS system, such as VMS systems 102B or 102C. Each of VMS systems 102B and 102C may operate a different communication or network interface. In an embodiment, interoperability gateway 118 may transmit the multimedia communications to interoperability gateways or console applications residing on separate network 105, such as a FirstNet network, or within, for example, VMS system 102B. Each of separate network 105 and VMS system 102B may be under the control or administration of an agency or operator different from that administering VMS system 102A.
Additionally, there may exist two or more networks of communications or agencies, each having different secrecy classifications. In an embodiment, interoperability gateway 118 may be coupled to or include a cross-domain information sharing system. Interoperability gateway 118 may validate a video feed's sharing permissions based on metadata associated with or embedded within the video feed. Upon validation, interoperability gateway 118 may retransmit or authorize transmission of the video feed to VMS systems 102 or across other networks.
Database 115 may be any type of structured data store, including a relational database that stores information associated with entities using the VMS application provided by VMS system 102A or associated with entities being controlled through VMS system 102A. For example, information associated with using entities may include information or parameters of operators and associated computing devices 124, and field users and associated PTT mobile devices 120. Information associated with controlled entities may include information or parameters of talk groups 122 and associated members, and controls or metadata of video cameras 110.
NVR 117 may be one or more video storage devices having application software for storing, searching, and retrieving stored video files. In an embodiment, NVR 117 may store video files in association with meta data tags, keywords, and other associated information files that are relationally associated with the video file and/or which are embedded as meta data within the video format and stored file. For example, other associated information associated with the video file may include attributes, such as a talk group or users, that are assigned to video camera 110 providing the video feed stored as the video file.
Therefore, NVR 117 may enable an operator accessing the VMS application to not only view a live video feed but also retrieve stored video feeds of video cameras 110 based on one or more search criteria. In an embodiment, an operator may input a talk group as a search criterion within the VMS application managed by VMS-PTT application server 116. NVR 117 may receive the search criterion and query the stored video files based on the search criterion to locate one or more video files associated with the search criterion. In an embodiment, NVR 117 may be representative of a system of devices, each of which may be implemented on one or more processors.
In an embodiment, NVR 117 or VMS-PTT application server 116 may additionally provide a video editing application that allows an operator of the VMS application to edit the recalled/retrieved video file and draw additional information through a graphical overlay via an editing application and/or edit or write additional metadata. The additional metadata, edits, or drawn information may be stored by NVR 117 or database 115.
PTT mobile devices 120 may be a mobile computing device (e.g., a smart phone, a tablet, a personal digital assistant (PDA), or a laptop computer) that includes a PTT client application or instance of the VMS application for accessing the VMS application provided or hosted by VMS system 102. In an embodiment, a field user may interface with PTT mobile device 120 through a VMS GUI provided through the PTT client. In an embodiment, a PTT client on, for example, PTT mobile device 120A may have access to the capabilities of PTT mobile device 120A that may include a touchscreen, a camera, a scanner, a text recognition interface, a location determiner (e.g., a Global Positioning System (GPS)), a speech recognizer, or a voice recorder. Information generated from these capabilities such as location information in space and time may be transmitted to other field users through PTT clients installed on respective PTT mobile devices 120, to VMS systems 102, or operators of monitoring devices 124.
Monitoring device 124 may be one or more stationary devices at a control monitoring center or station that are used by one or more operators to monitor one or more video cameras 110. An operator may interface with a PTT client or VMS application instance on the monitoring device 124 to access the video feeds provided by video cameras 110. In an embodiment, the operator need not use monitoring device 124 to control or access or operate the VMS application provided by VMS system 102A. Instead, an operator may be a field personnel using one or more of PTT mobile devices 120.
In an embodiment, VMS-PTT application server 202 may include various modules (or components) to implement the functionality of the provided VMS GUI. For ease of understanding, descriptions of the modules of
Monitoring module 201 may provide to PTT clients on PTT mobile devices 120 or monitoring devices 124 respective VMS GUIs such that an operator may efficiently communicate to select field personnel operating respective PTT clients. In an embodiment, monitoring module 201 may provide to an operator of monitoring device 124A the example VMS GUI view 300 of
In an embodiment, monitoring module 201 may provide an operator access to, for example, four distinct video feeds displayed within corresponding video feed components 308A-D of respective IMMOs 302A-D of VMS GUI view 300. Each video feed 308 may originate from a corresponding video camera 110, which is designated by video camera ID 309. Video camera ID 309 may be an IP address or any sequence of characters that uniquely identifies video camera 110 providing the video feed displayed by video feed component 308. In an embodiment, monitoring module 201 may enable the operator to select the number of IMMOs 308 and one or more specific video feeds to be displayed within corresponding IMMOs 308 depending on attributes of the operator or video cameras 110 as further described. In an embodiment, the selected one or more video feeds may be transmitted by monitoring module 201 to each member of talk groups coupled to the corresponding one or more video feeds.
Though the operator has been described as being human, in an embodiment, the operator may be human with machine assistance or entirely machine-based. For example, the operator may be a video analytics module within monitoring component 202 that selects specific video feeds to be displayed within IMMOs 302. The video analytics module may interpret and process video images from video feeds captured by video cameras 110 and select a subset of video feed views to monitor based on rules and parameters configured by a human operator or administrator. In an embodiment, the video analytics module may use the rule or parameters to implement algorithms, dictionaries, neural nets, or statistical or heuristic methods to recognize or detect objects, events, or patterns of interest to the human operator or administrator. An entirely or partially machine-assisted operator may be further defined by parameter and rules for determining talk group(s) and initiating PTT communications with the determined talk group(s). Additional examples of rules or parameters include associated sensor information alerts, visual or electronic distress signals, or detected beacon activation.
In an embodiment, the video analytics module may be implemented on a computer processing chip of video camera 110, on a module embedded or coupled to video camera 110, on a computer processing device remotely executing software that is coupled by a communications network to video camera 110, or on a combination of any module or device listed. In an embodiment, upon detecting an event or pattern of interest captured by, for example, video camera 110A, monitoring module 201 may notify a human operator or prompt the operator to select the video feed provided by video camera 110A to be displayed within a IMMO, such as IMMO 302A of VMS GUI 300.
In an embodiment, monitoring module 201 may provide independent VMS GUIs within PTT clients to corresponding users, such as field personnel or operators of PTT mobile devices 120 of
In an embodiment, monitoring module 201 may include a video coding module to improve a quality of video feeds. The video coding module may include coding functions to control the video feed by decoding, re-encoding or transcoding, storing and forwarding, injecting with metadata. In an embodiment, these coding functions may be implemented on within VMS-PTT application server 202, mobile devices 120, monitoring device 124, or a separate media server application within VMS system 102A. Monitoring module 201 may implement transmission of video feeds to and from PTT clients via UDP, TCP or other data transmission protocol and may employ packet stream replication, packet relay, multicasting, broadcasting, or other transmission means.
Permissions module 204 may assign attributes to users, such as field personnel and operators, using PTT mobile devices 120 or monitoring device 124. An assigned attribute of a user may include any of the following: an identity, a job title, one or more talk groups, position, one or more group, one or more task, one or more video camera, one or more agency or group affiliations, one or more access privilege status, or one or more priority settings. The assignments may be received from, for example, an administrator, a human resources manager, or the user's supervisor. In an embodiment, permissions module 204 may store the attributes of the user and permission settings associated with each attribute in database 115 of
In an embodiment, priorities associated with an assignment may be used in determining an order or sequence to execute concurrent commands from a plurality of authenticated and authorized members of a talk group. Concurrent commands may include, for example, sending and receiving of voice messages.
For example, an operator may have a role attribute indicating the operator's monitoring task and field personnel may be assigned a role attribute indicating a first responder task. Therefore, if an operator and a field personnel send concurrent voice messages, the role attributes may enable the operator[[s]] to have a higher priority when sending voice messages.
In an embodiment, an attribute may be dynamically assigned. For example, a dynamically assigned attribute of a user may include a dynamic talk group, a geolocation of user based on an associated PTT mobile device 120, or whether the user is within proximity of video camera 110. In an embodiment, permissions module 204 may determine whether the user is within proximity of any video camera 110 based on the assigned dynamic talk group, or the geolocation attribute and received or stored geolocations of each video camera 110. Relatedly, in an embodiment, permissions for accessing video cameras 110 may be dynamically updated based on dynamically assigned attributes.
In an embodiment, if a module, such as monitoring module 201, requests access to video camera 110A based on operator input (which may be machine-assisted), permissions module 204 may query database 115 to determine whether the operator is authorized to access video camera 110A. For example, permissions module 204 may associate an attribute of “John Smith” (an identity) and “Area 1” (a group ID) with video camera 110C such that the operator or user is authorized if the user is “John Smith” or is assigned a group ID of “Area 1.”
Talk group module 206 may manage, establish, or assign talk groups based on user attributes or video camera attributes. As discussed, a talk group is a logically defined voice or PTT communications group that allows members to send and receive voice messages to and from other member users in the talk group. In an embodiment, talk group module 206 may store current members of an established talk group within one or more talk group tables within database 115.
In an embodiment, a talk group may be defined or pre-assigned by a user including a field user or an operator using talk group component 312 depicted in
In an embodiment, talk group module 206 may be requested by an operator to establish a dynamic talk group based on a dynamically assigned attribute of a user. For example, talk group module 206 may receive a proximity threshold attribute to be associated with video camera 110A providing, for example, the video feed of video feed component 308A. Talk group module 206 may query database 115 to determine users or field personnel having a proximity-based attribute indicating that one or more of their PTT mobile devices 120 are within the received proximity threshold attribute. In an embodiment, talk group module 206 may be requested to re-evaluate whether a user should continue to be a member of the talk group when the proximity-based attribute changes. If the user no longer satisfies the attribute of the talk group, talk group module 206 may dynamically remove the user as a member of the established talk group.
As described, talk group module 206 may dynamically determine members to be added to the established talk group based on a dynamically or statically assigned attributes of users. For example, talk group module 206 may determine a member of the talk group based on whether a distance between geolocations, which are dynamically assigned attributes, of PTT mobile devices 120 operated by respective users and video camera 110A are within a proximity threshold. In an embodiment, talk group module 206 may receive geolocations of PTT mobile device from PTT mobile device 120 or another device having location-determining capabilities and coupled to PTT mobile device 120. In an embodiment, the device having location-determining capabilities, such as PTT mobile device 120 having a GPS, may determine a distance between the device and one or more video cameras 110. Then, the device may send the determined distance to be used by talk group module 206 to determine proximity of the device to the one or more video cameras 110.
In an embodiment, an attribute of a user may be whether the user's PTT mobile device 120 is within a communication range of a beacon or has been within range for a predetermined period of time. A beacon may be a sensor or other wireless communication device that may be associated with or placed near, for example, video camera 110A. In an embodiment, PTT mobile device 120 may receive from the beacon a wireless signal or information that furnishes location or other information from which location of the beacon or an approximate location of PTT mobile device 120 or other device coupled to PTT mobile device 120 can be determined. Therefore, the attribute may be assigned based on beacon information indicating location, GPS information of mobile device 120, or a combination thereof.
In an embodiment, talk group module 206 may additionally or alternatively determine that users are within proximity of video camera 110A based on information gathered by video camera 110A. For example, the user may be within a proximity threshold of video camera 110A if the user is visually detected and recognized within a field of view of video camera 110A within a predetermined period of time. In an embodiment, an analytics module implemented in VMS-PTT application server 116 or in video camera 110A may identify the user based on detectable light emission parameters of the video feed. The parameters include facial pattern recognition, infrared beacon associated with the user, or a wearable visual identifier such as symbol or color or pattern.
Talk group coupling module 206 may logically associate a talk group with a video feed provided by a video camera by coupling the talk group to an embedded PTT component associated with the video feed or video camera. In an embodiment, talk group coupling module 206 logically associates each talk group coupling component 304 with components of corresponding IMMOs 302: embedded PTT control component 306, video feed component 308, and video camera ID component 309. In an embodiment, talk group coupling module 206 may associate a talk group with an IMMO 302 including more than one video feed component 308. In the embodiment shown in VMS GUI view 300, talk group coupling module 206 may logically and visually associate, for example, the talk group “TG-1 Security” with video feed component 308A or embedded PTT control component 306A within IMMO 302A.
In an embodiment, talk group coupling module 206 selects the talk group to be coupled to an embedded PTT control component based on an input received from an operator from talk group coupling component 304 depicted in VMS GUI view 300. In this embodiment, talk group coupling component 304 acts as a selector control component for selecting one or more talk groups. For example, talk group coupling component 304C may provide a list of talk groups from which the operator may select to logically and visually associate with video feed component 308C. As shown, talk group “TG-Counter” was selected and talk group coupling module 206 couples the talk group “TG-Counter” to embedded PTT control component 306C.
In an embodiment, a talk group may be selected from talk groups established by talk group module 206, which may depict a subset of established talk groups within talk group component 312. In an embodiment (not shown), talk group coupling component 304 may enable an operator to select more than one talk group to be logically or visually associated with one or more video feed components 308.
In an embodiment, upon logically associating a talk group with a video feed, talk group coupling module 206 may update one or more attributes, or update or grant permissions of each member of the talk group with respect to one or more components of IMMO 302 containing the video feed. Talk group coupling module 206 may update database 115 directly or request permissions module 204 to perform the updates. For example, upon logically associating talk group “TG-2-Warehouse” with video feed component 308C, talk group coupling module 206 may grant each member access to the video feed displayed within video feed component 308B and prioritized remote control of video camera 110B designated by video camera ID 309B.
PTT control module 210 may detect that an embedded PTT control component, such as embedded PTT component 306A, associated with a video feed, such as a video feed displayed by video feed component 308A, is being or has been activated. Upon detection, PTT control module 210 may enable PTT communications regarding content displayed by a video feed between an operator and each member of a talk group, such as “TG-1 Security” displayed by talk group coupling component 304A, coupled to the video feed. PTT communications may include PTT voice functions and commands distributed across one or more networks, such as IP network 104 and radio-mobile network 106.
In an embodiment, to more effectively integrate PTT two-way communication into VMS system 102, PTT control module 210 may present VMS GUI view 300 that couples one embedded PTT control component 306 for each IMMO 308, which may each include one (as shown) or more (not shown) video feeds.
In an embodiment, PTT control module 210 may enable various types of communication resources to be transmitted by one or more members to each other member within a talk group. For example, such communication resources may include data generated by a member operating, for example, PTT mobile device 120A. In an embodiment, the data generated may include at least one of the following: video feed, images, readable files, audio messages, and text messages. PTT control module 210 may display components to send and display or execute the generated data within corresponding IMMOs of VMS GUI view 300.
Video camera controller module 212 may control one or more video cameras 110 managed by VMS-PTT application server 202 based on a received remote command. In an embodiment, video camera controller module 212 may receive the remote command for a video camera, such as video camera 110E, from an operator of PTT mobile device 120B. In an embodiment, the remote command may affect a metadata captured by video camera 110. For example, the remote command may be a direction, inclination, angle, moment, velocity, or acceleration, tilt, pan, or zoom of video camera 110.
In an embodiment, upon receiving the remote command, video camera controller module 212 may request permissions module 204 to authorize the operator or user. If the user is authorized, video camera controller module 212 may transmit an electronic request to the corresponding video camera 110 to perform the remote command. In an embodiment, multiple commands may be concurrently received by multiple operators. In such a scenario, remote commands of respective users may be prioritized based on one or more attributes assigned to users. For example, it may be more beneficial for an operator using PTT mobile device 120C that is within proximity of, for example, video camera 110B to be given higher control priority than another operator that is not within proximity of video camera 110B.
Geographic information system (GIS) module 214 may present a map interface within a VMS GUI provided by PTT clients or VMS application instances operated by PTT mobile devices 120 or monitoring devices 124. In an embodiment to be further described in
Interoperability gateway module 216 may communicate with interoperability gateway 118 of
In an embodiment, an operator using a device and viewing the VMS GUI view of
In an embodiment, a PTT client on mobile device 120 and providing one or more of VMS GUI of
As shown by the poor quality of the video feed shown in
In an embodiment, regardless of the quality of video feed shown in
Similar to
Video sending component 510, when requested by an operator, may distribute view of video feed component 508 to each member of the talk group, such as “TG: −2 Patrol” if video sending component 510A is enabled, selected within talk group coupling component 504.
Proximate members components 512, when requested by the operator, may show or reveal members of a talk group within proximity of the video camera providing video feed component 508. In an embodiment, the range of proximity may be set by proximity range component 516 based on operator input or a proximity threshold attribute assigned to the video camera. In an embodiment, proximate members component 512 may be representative of an always active dynamic talk group having members that are proximate to corresponding video cameras.
In an embodiment, IMMO 502 may execute an audio alert, a visual alert, a visual designation, or a combination thereof to indicate the proximate presence, or non-presence, of a member of an associated talk group in relation to a video camera. The talk group may be based on the talk group coupling component 504, proximate members component 512, or both. In an embodiment, other information associated with talk group members including heart rate, blood pressure, perspiration, or acceleration may be visually displayed or represented within IMMOs 502 or another window, frame, or object associated with respective IMMOs 502.
Video camera location icons 514 may be a visual icon representative of respective IMMOs 502 that are smaller than IMMOs 502 and obscures as little of map 502 as possible. Video camera location icons 514 may be selected and de-selected to maximize or minimize IMMOs 502, respectively.
In an embodiment, the grouping of components of IMMOs 502 and components' or IMMOs' presence and relative positioning within map 502 of VMS GUI view 500 may be organized based on rules or parameters. For example, whether specific components of IMMOs 502 available and orientation of video camera feed components 508 may be based on priority attributes associated with an operator viewing VMS GUI 500. In an embodiment, video camera feeds and associated IMMOs 502 may be prioritized, for example, enlarged, based on video camera attributes, such as image definition of video camera feed, functions or capabilities of the video cameras, or meta data generated from a video camera such as direction, altitude, speed, or detection of object of interest by a video analytic module.
Method
In step 602, a monitoring module of the VMS-PTT application server, such as monitoring module 201 of
In an embodiment, the monitoring module presents video feed views of video cameras selected by an operator within video feed components of corresponding IMMOs displayed in the VMS GUI. For example, the exemplary VMS GUI view 300 of
In step 604, a talk group coupling module of the VMS-PTT application server, such as talk group coupling module 208 of
In an embodiment, the operator may select a defined talk group to be logically associated with the video feed or a group of video feeds including the video feed. The selected talk group may be a talk group that is predetermined or dynamically established by a talk group module of the VMS-PTT application server, such as talk group module 206 of
In step 606, a PTT control module, such as PTT control module 206 of
In step 608, the PTT control module enables PTT communications with each member of the talk group regarding content displayed by the video feed logically associated with the talk group. In an embodiment, the operator may be considered an automatic member of the talk group. Upon detecting, in step 606, that the embedded PTT control component is activated, the PTT control module enables PTT communication between each member of the talk group coupled to the embedded PTT control component.
In step 702, a permissions module of the VMS-PTT application server, such as permissions module 204 of
In step 704, a talk group module, such as talk group module 206 of
In step 706, the talk group module determines whether one or more field personnel is proximate to the video camera and therefore satisfies the proximate component. A field personnel may be proximate to the video camera if within a defined proximity threshold. In an embodiment, the talk group module may query attributes assigned by the permissions module to determine a field personnel's proximity to the video camera. For example, the permissions module may assign a geolocation attribute to a field personnel based on a received geolocation of the field personnel from his mobile device or a geolocation of a device detecting the field personnel, such as a WiFi access point connecting the field personnel to a network. Using the geolocation attributes, the talk group module may compare the geolocations between the field personnel and the video camera to determine whether the field personnel is within the proximity threshold. In an embodiment, the VMS may execute facial recognition or personal ID software on the contents captured by the video feed to detect the field personnel. If the field personnel is being captured by the video feed, the field personnel may be determined as proximate to the video feed.
In step 708, if the field personnel is proximate to the video camera, method 700 proceeds to step 710. If the field personnel fails the proximate component of the dynamic access rule, method 700 proceeds to step 716.
In step 710, the talk group module determines whether the field personnel is permitted to access the video camera and therefore satisfies the access component. In an embodiment, the talk group module may query attributes assigned by the permissions module to determine a field personnel's permissions or lack thereof for accessing or controlling one the video feed of the video camera.
In step 712, if the field personnel is verified, method 700 proceeds to step 714. If the field personnel fails the access component of the dynamic access rule, method 700 proceeds to step 716. In an embodiment, the talk group module may electronically transmit an invitation to PTT mobile unit users who satisfy the dynamic access rule to join the dynamic talk group. An exemplary invitation process is further explained in patent application Ser. No. 14/609,815.
In step 714, the talk group module may add the field personnel satisfying the dynamic access rule to the dynamic talk group associated with the video feed. In an embodiment, if any component of the dynamic access rule is updated, steps 706-712 may need to be re-performed for each current member of the dynamic talk group. In an embodiment, the geolocation attribute of a field personnel may be updated. In this case, steps 706-712 may need to be re-performed for that field personnel.
In step 716, if the field personnel does not satisfy the dynamic access rule either by failing the proximity component in step 708 or the access component in step 712, the talk group module removes or confirms that the field personnel is not a member of the dynamic talk group. In an embodiment, if a current member of the dynamic talk group is determined to no longer satisfy the dynamic access rule, the talk group module may remove the current member from the dynamic talk group.
System Implementation
Various embodiments can be implemented, by software, firmware, hardware, or a combination thereof.
Computer system 800 includes one or more processors (also called central processing units, or CPUs), such as a processor 804. Processor 804 is connected to a communication infrastructure or bus 806.
One or more processors 804 may each be a graphics processing unit (GPU). In an embodiment, a GPU is a processor that is a specialized electronic circuit designed to process mathematically intensive applications. The GPU may have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc.
Computer system 800 also includes user input/output device(s) 803, such as monitors, keyboards, pointing devices, etc., that communicate with communication infrastructure 806 through user input/output interface(s) 802.
Computer system 800 also includes a main or primary memory 808, such as random access memory (RAM). Main memory 808 may include one or more levels of cache. Main memory 808 has stored therein control logic (i.e., computer software) and/or data.
Computer system 800 may also include one or more secondary storage devices or memory 810. Secondary memory 810 may include, for example, a hard disk drive 812 and/or a removable storage device or drive 814. Removable storage drive 814 may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.
Removable storage drive 814 may interact with a removable storage unit 818. Removable storage unit 818 includes a computer usable or readable storage device having stored thereon computer software (control logic) and/or data. Removable storage unit 818 may be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/any other computer data storage device. Removable storage drive 814 reads from and/or writes to removable storage unit 818 in a well-known manner.
According to an exemplary embodiment, secondary memory 810 may include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system 800. Such means, instrumentalities or other approaches may include, for example, a removable storage unit 822 and an interface 820. Examples of the removable storage unit 822 and the interface 820 may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.
Computer system 800 may further include a communication or network interface 824. Communication interface 824 enables computer system 800 to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference number 828). For example, communication interface 824 may allow computer system 800 to communicate with remote devices 828 over communications path 826, which may be wired and/or wireless, and which may include any combination of LANs, WANs, the Internet, etc. Control logic and/or data may be transmitted to and from computer system 800 via communication path 826.
In an embodiment, a tangible apparatus or article of manufacture comprising a tangible computer useable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system 800, main memory 808, secondary memory 810, and removable storage units 818 and 822, as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system 800), causes such data processing devices to operate as described herein.
Based on the teachings contained in this disclosure, it will be apparent to persons skilled in the relevant art(s) how to make and use embodiments of the invention using data processing devices, computer systems and/or computer architectures other than that shown in
It is to be appreciated that the Detailed Description section, and not the Summary and Abstract sections (if any), is intended to be used to interpret the claims. The Summary and Abstract sections (if any) may set forth one or more but not all exemplary embodiments of the invention as contemplated by the inventor(s), and thus, are not intended to limit the invention or the appended claims in any way.
While the invention has been described herein with reference to exemplary embodiments for exemplary fields and applications, it should be understood that the invention is not limited thereto. Other embodiments and modifications thereto are possible, and are within the scope and spirit of the invention. For example, and without limiting the generality of this paragraph, embodiments are not limited to the software, hardware, firmware, and/or entities illustrated in the figures and/or described herein. Further, embodiments (whether or not explicitly described herein) have significant utility to fields and applications beyond the examples described herein.
Embodiments have been described herein with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined as long as the specified functions and relationships (or equivalents thereof) are appropriately performed. Also, alternative embodiments may perform functional blocks, steps, operations, methods, etc. using orderings different than those described herein.
The breadth and scope of the invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.
Claims
1. A system for video management with Push To Talk (PTT) communications, comprising:
- a memory;
- one or more processors coupled to the memory, and configured to: monitor a first video feed using a video management software (VMS) graphics user interface (GUI), wherein the first video feed is provided by a first video camera, wherein the first video feed is a component of an interactive multimedia object (IMMO) displayed in the VMS GUI; pre-assign a first talk group and a second talk group to the first video camera; associate the first talk group with the first video feed; transmit the first video feed to members of the first talk group; receive a selection of the second talk group; transmit the first video feed to members of the second talk group; monitor a plurality of video feeds including the first video feed, wherein each video feed of the plurality of video feeds is provided by a corresponding video camera of a first plurality of video cameras, wherein a first plurality of video feeds of the plurality of video feeds are components of the IMMO displayed in the VMS GUI; transmit a first selection of the first plurality of video feeds to members of the second talk group; receive a second selection of the first plurality of video feeds; and transmit the second selection to members of the second talk group.
2. The system of claim 1, wherein the one or more processors are further configured to:
- detect that an embedded PTT control component associated with the first video feed is activated; and
- enable PTT communications among members of the second talk group and an operator of the VMS GUI.
3. The system of claim 1, wherein the second selection is received from an operator of the VMS GUI.
4. The system of claim 1, wherein the second selection is received from a member of the second talk group, the one or more processors are configured to: grant permission to the member of the second talk group.
5. The system of claim 4, wherein the permission is based on rules or parameters.
6. The system of claim 4, wherein the one or more processors are further configured to:
- receive a third selection of a third talk group from the member of the second talk group; and
- enable PTT communications among members of the third talk group and an operator of the VMS GUI.
7. The system of claim 1, wherein the one or more processors are further configured to resize the one or more video feeds of the second selection.
8. The system of claim 1, wherein the one or more processors are further configured to:
- display a plurality of IMMOs including the IMMO on a map on the VMS GUI; and
- correlate geolocations of a second plurality of video cameras including the first plurality of video cameras with respective IMMOs of the plurality of IMMOs.
9. The system of claim 8, wherein the one or more processors are further configured to:
- present proximate members of the second talk group on the IMMO, wherein the proximate members are members of the second talk group that satisfy a proximity threshold attribute associated with a corresponding video camera of the IMMO.
10. A non-transitory computer-readable medium having instructions stored therein, which when executed by one or more processors in an electronic device cause the one or more processors to perform operations for video management with Push To Talk (PTT) communications, the operations comprising:
- monitoring a first video feed using a video management software (VMS) graphics user interface (GUI), wherein the first video feed is provided by a first video camera, wherein the first video feed is a component of an interactive multimedia object (IMMO) displayed in the VMS GUI;
- pre-assigning a first talk group and a second talk group to the first video camera;
- associating the first talk group with the first video feed;
- receiving a selection of the second talk group wherein members of the second talk group access the first video feed;
- monitoring a plurality of video feeds including the first video feed, wherein each video feed of the plurality of video feeds is provided by a corresponding video camera of a first plurality of video cameras, wherein a first plurality of video feeds of the plurality of video feeds are components of the IMMO displayed in the VMS GUI;
- transmitting a first selection of the first plurality of video feeds to members of the second talk group;
- receiving a second selection of the first plurality of video feeds; and
- transmitting the second selection to members of the second talk group.
11. The non-transitory computer-readable medium of claim 10, wherein the operations further comprise:
- detecting that an embedded PTT control component associated with the first video feed is activated; and
- enabling PTT communications among members of the second talk group and an operator of the VMS GUI.
12. The non-transitory computer-readable medium of claim 10, wherein the second selection is received from an operator of the VMS GUI.
13. The non-transitory computer-readable medium of claim 10, wherein the second selection is received from a member of the second talk group, the one or more processors are configured to:
- grant permission to the member of the second talk group, wherein the permission is based on rules or parameters.
14. The non-transitory computer-readable medium of claim 13, wherein the operations further comprise:
- receiving a third selection of a third talk group from the member of the second talk group; and
- enabling PTT communications among members of the third talk group and an operator of the VMS GUI.
15. A method for video management with Push To Talk (PTT) communications, comprising:
- monitoring a first video feed using a video management software (VMS) graphics user interface (GUI), wherein the first video feed is provided by a first video camera, wherein the first video feed is a component of an interactive multimedia object (IMMO) displayed in the VMS GUI;
- pre-assigning a first talk group and a second talk group to the first video camera;
- associating the first talk group with the first video feed;
- receiving a selection of the second talk group;
- coupling the second talk group with the first video feed, wherein members of the first talk group no longer access the first video feed;
- monitoring a plurality of video feeds including the first video feed, wherein each video feed of the plurality of video feeds is provided by a corresponding video camera of a first plurality of video cameras, wherein a first plurality of video feeds of the plurality of video feeds are components of the IMMO displayed in the VMS GUI;
- transmitting a first selection of the first plurality of video feeds to members of the second talk group;
- receiving a second selection of the first plurality of video feeds; and
- transmitting the second selection to members of the second talk group.
16. The method of claim 15, further comprising:
- detecting that an embedded PTT control component associated with the first video feed is activated; and
- enabling PTT communications among members of the second talk group and an operator of the VMS GUI.
17. The system of claim 1, wherein the first video feed and a second video feed of the plurality of video feeds correspond to the first video camera and a second video camera of the first plurality of video cameras, the one or more processors are further configured to:
- prioritize the IMMO corresponding to the first video feed and a second IMMO corresponding to the second video feed based at least on video camera attributes of the first and the second video cameras, wherein the first and second video cameras are managed through a VMS server, wherein the first video camera is fixedly coupled to an intercom with a speaker and microphone.
18. The system of claim 17, wherein video camera attributes of the first video camera comprises an image definition of the first video feed, or a capability of the first video camera.
19. The system of claim 17, wherein video camera attributes of the first video camera comprise meta data generated by the first video camera including: a direction, altitude, speed, or detection of an object of interest.
20. The method of claim 15, wherein the first video feed and a second video feed of the plurality of video feeds correspond to the first video camera and a second video camera of the first plurality of video cameras, the method further comprises:
- prioritizing the IMMO corresponding to the first video feed and a second IMMO corresponding to the second video feed based at least on video camera attributes of the first and the second video cameras, wherein the first and second video cameras are managed through a VMS server, wherein the first video camera is fixedly coupled to an intercom with a speaker and microphone.
| 8581957 | November 12, 2013 | Bengtsson |
| 9313556 | April 12, 2016 | Borel |
| 9787940 | October 10, 2017 | Mazzarella et al. |
| 10075674 | September 11, 2018 | Mazzarella et al. |
| 20050143135 | June 30, 2005 | Brems |
| 20060121924 | June 8, 2006 | Rengaraju |
| 20070054686 | March 8, 2007 | Allen |
| 20070239824 | October 11, 2007 | Shaffer |
| 20080108339 | May 8, 2008 | Shaffer et al. |
| 20100033580 | February 11, 2010 | Shaffer et al. |
| 20100159976 | June 24, 2010 | Marocchi et al. |
| 20110143651 | June 16, 2011 | Marocchi |
| 20110154446 | June 23, 2011 | Drozt |
| 20110225238 | September 15, 2011 | Shaffer et al. |
| 20120265867 | October 18, 2012 | Boucher |
| 20120313755 | December 13, 2012 | Gutierrez |
| 20130283330 | October 24, 2013 | Hengeveld |
| 20140187280 | July 3, 2014 | Bekiares |
| 20140368601 | December 18, 2014 | deCharms |
| 20160219628 | July 28, 2016 | Sharma |
| 20160227084 | August 4, 2016 | Imamura et al. |
| 20160227384 | August 4, 2016 | Mazzarella |
| 20170099455 | April 6, 2017 | Mazzarella et al. |
| 20180035078 | February 1, 2018 | Mazzarella et al. |
- International Search Report and Written Opinion for PCT Application No. PCT/US2016/055544, filed Oct. 5, 2016, 13 pages, dated Jan. 18, 2017.
- International Report on Patentability for PCT Application No. PCT/US2016/055544, filed Oct. 5, 2016, 8 pages, dated Apr. 10, 2018.
- Office Action directed to related European Patent Application No. 16788289.3, dated Feb. 7, 2019; 11 pages.
- Office Action directed to related European Patent Application No. 16788289.3, dated Nov. 7, 2019; 10 pages.
- Office Action directed to related Australian Patent Application No. 2016333974, dated Feb. 28, 2020; 3 pages.
Type: Grant
Filed: Sep 7, 2018
Date of Patent: Dec 29, 2020
Patent Publication Number: 20190007650
Assignee: Mutualink, Inc. (Wallingford, CT)
Inventors: Joseph R. Mazzarella (Tolland, CT), Michael S. Wengrovitz (Concord, MA), Mark Hatten (Farmington, CT)
Primary Examiner: Maria El-Zoobi
Application Number: 16/125,569
International Classification: H04M 1/2745 (20200101); H04N 7/14 (20060101); H04W 4/90 (20180101); H04W 4/029 (20180101); H04L 29/06 (20060101); H04W 4/08 (20090101); H04L 29/08 (20060101); H04W 4/10 (20090101); H04W 4/14 (20090101);
