Abstract: A method, smart wearable device and computer program product for detecting inter-person conversations. Audio data is captured on the smart wearable device. Such audio data may be from various sources, including sounds that only involve listening by an individual (e.g., television show) and sounds that involves an individual participating in face-to-face communication. Acoustic features are then extracted from the captured audio data. Such extracted acoustic features are a description of the captured audio data. Neural network embedding features are then extracted from the extracted acoustic features using a first neural network model. The extracted neural network embedding features are then fused into a second neural network model (configured to detect inter-person conversations based on detecting speaker change point(s)) to perform user conversation inference. In this manner, inter-person conversations are more effectively detected using a smart wearable device.