Beyond the Question: Reading Faces and Voices in Research
#biometrics#voice#facial recognition#OAIRA#qualitative research#labs
David Olsson
There's a class of research signal that questionnaires cannot capture.
The 97% confidence score on a neutral expression. The 43% spike in vocal energy at exactly the moment a respondent mentions a competitor. The hesitation — 2.8 seconds of silence — before answering a question about switching intent.
This signal exists in every research conversation. Traditional instruments throw it away. OAIRA's biometric labs capture it.
Facial Recognition Lab
The Facial Recognition Lab provides real-time face detection, expression analysis, and identity recognition via camera input.
Detection capabilities:
- Face detection — Detects faces in the camera frame with configurable confidence thresholds
- Landmarks — Facial landmark mapping for precise expression tracking
- Expressions — Real-time emotional expression classification: Neutral, Happy, Sad, Angry, Surprised, Fearful
- Age & Gender — Demographic inference from facial features
- Recognition — Identity matching against registered faces
- Hand tracking and Body pose — Extended presence signal capture
Live detection output includes:
- Current expression with confidence score (e.g., Neutral 97%)
- Match score for registered identities
- Landmark data for granular expression mapping
The confidence score is shown live, updating in real time as the respondent's expression changes. This isn't a post-session analysis. It's a live feed of emotional state.
Expression targeting lets researchers configure a target expression and track when and how often a respondent's actual expression matches it. Running a concept test and want to know when respondents genuinely react? Set a target expression and the lab tracks it.
Voice Recognition Lab
The Voice Recognition Lab analyzes live audio for acoustic features and emotional state in real time.
Analysis timeline shows layered waveform data:
- Waveform — Raw audio amplitude over time
- Acoustic features — Pitch (red), Volume (cyan), Energy (yellow) tracked continuously
- Emotions — Emotional state confidence levels over time, visualized as a timeline
- Transcript — Live speech-to-text with timing
- Markers — Annotatable points in the timeline for researcher notes
Live metrics:
- Current emotion with confidence percentage
- Pitch in Hz and volume in dB
- Energy level and spectral centroid
- Speech rate in WPM
- Zero crossing rate (indicator of voice texture and stress)
- Voice activity detection and silence duration
Session statistics summarize the full session: average pitch, average volume, total speech duration, dominant emotional state.
Emotion detection classifies: Neutral, Sad, Surprised, Angry, Calm, Fearful, Happy, Excited — with confidence scores for each state at each moment in the recording.
What This Changes About Research
The conventional view of market research data is structured response data: ratings, selections, text. The respondent produces data by consciously deciding what to submit.
Biometric data is different. It captures what the respondent didn't decide to say. It captures the involuntary — the expression that crosses a face before words form, the vocal stress that accompanies an answer that doesn't quite match the score on the scale.
This creates several new research capabilities:
Verification. When a respondent rates their satisfaction as 8/10 but their vocal energy spikes into stress patterns at the moment they answer, that's information. The biometric doesn't replace the survey response; it contextualizes it.
Reaction tracking. Show a respondent a stimulus — an ad concept, a product prototype, a pricing structure — and capture facial and acoustic response in real time. The question "what did you think?" can now be supplemented by the answer their face gave before they could formulate a verbal response.
Engagement monitoring. Vocal energy and facial expression patterns correlate with engagement. A respondent who maintains high energy and varied expression throughout an interview is more engaged than one whose metrics flatline. This helps calibrate how much to weight data from each session.
Emotional trajectory. The timeline view shows how emotional state evolves over the course of a conversation. Which topics produced stress? Which produced enthusiasm? The pattern across the session is often more informative than any individual data point.
Research Ethics and Consent
Biometric data collection in research requires explicit informed consent. OAIRA's biometric labs are researcher-facing tools — they run in the research environment, not silently in the background of respondent interfaces.
The camera and microphone activation are clearly indicated. Registered faces require explicit registration. The data stays within the platform's security boundary.
These are powerful capabilities. Like all powerful capabilities, they require careful, consent-based application.
A New Layer of Signal
The question "what do people think?" has always been harder to answer than it looks. People don't always know what they think. They know what they're willing to say. They know what sounds socially acceptable. They know what the question seemed to be asking for.
Facial expressions and voice acoustics bypass some of these filters. They're not perfect — expression and acoustic patterns are probabilistic, not deterministic. But as a complement to structured response data, they add a layer of signal that questionnaires alone cannot provide.
OAIRA's biometric labs make that signal available. What researchers do with it is the interesting question.
OAIRA is an AI-powered market research platform. The Facial Recognition Lab and Voice Recognition Lab are available in the Labs section. All biometric capture requires explicit research consent protocols.