7 Questions OEMs Should Ask Before Commissioning a Custom rPPG Model

The decision by a hardware original equipment manufacturer (OEM) to move beyond generic, one-size-fits-all software and commission a custom remote photoplethysmography (rPPG) model is a significant indicator of engineering maturity. It signals a shift from proof-of-concept to product-grade execution, where performance, reliability, and differentiation are critical. This move is driven by the realization that off-the-shelf models, while useful for initial prototyping, often fail to meet the stringent accuracy requirements of a specific camera, use case, and operating environment. Committing to a custom build is a substantial investment of time and resources, and the success of the project hinges on asking the right questions upfront.

"The market for digital health monitoring is projected to grow to over $850 billion by 2030, with camera-based sensing technologies representing a significant portion of this expansion. However, the accuracy of these systems is not a given; it is highly dependent on sensor-specific and use-case-specific model training." - Industry analysis from a 2023 report by Acumen Research and Consulting.

Key OEM questions before a custom rPPG model commission

For hardware OEMs, automotive suppliers, and IoT device makers, the process of developing a custom vital signs algorithm requires a methodical approach. The answers to these questions will define the scope, feasibility, and ultimate success of the integration. These are the critical oem questions before custom rppg model development begins, forming the foundation of a technical requirements document.

1. what are the exact camera, sensor, and optics specifications?

The performance of an rPPG model is inextricably linked to the "eyes" of the system. A model trained on a high-resolution, global shutter sensor will not perform optimally on a low-resolution, rolling shutter sensor common in low-cost IoT devices. Key details to define include:

• Sensor Type: CMOS, IR, or Thermal? Each has a distinct spectral response.
• Resolution and Frame Rate: Higher resolution is not always better if it compromises the frame rate needed to capture the blood volume pulse.
• Lens and Field of View (FOV): This determines the distance and perspective of the subject.
• Image Signal Processor (ISP) Pipeline: How are auto-exposure, auto-white balance, and color space transformations handled? These settings can dramatically alter the signal. Research from a team at the University of Oulu (2021) demonstrated that uncontrolled auto-exposure can introduce non-linearities that corrupt the rPPG signal.

2. what is the intended use case and operating environment?

The context in which the device will operate is as important as its hardware. A model for an automotive driver monitoring system must handle extreme lighting changes, vibrations, and head motion. A model for a clinical kiosk operates in a controlled, indoor environment but must account for a wide diversity of skin tones and skin conditions. Define the "domain" of operation:

• Lighting Conditions: Bright, dim, mixed, rapidly changing?
• Subject Distance and Motion: Is the subject stationary (kiosk), mostly stationary (automotive), or moving freely (smart home device)?
• Target Population: Will the device be used by a specific demographic group (e.g., elderly patients, neonatal infants)?

3. how will ground-truth data be acquired?

A model is only as good as the data it is trained on. For rPPG, this means synchronized video footage and ground-truth physiological data from a trusted contact-based sensor. The choice of this reference device is a critical decision. It must be a reliable, well-calibrated instrument that measures the same physiological parameter the rPPG model aims to estimate. For heart rate, this is typically an electrocardiogram (ECG) or a contact photoplethysmography (cPPG) finger clip.

4. how will model performance and accuracy be validated?

Defining success criteria is non-negotiable. Metrics must be agreed upon before development starts. Common statistical measures include Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and the Bland-Altman agreement analysis. The validation protocol should specify:

• The held-out test dataset: This data must be entirely separate from the training and validation sets.
• The diversity of the test dataset: It must include challenging scenarios and population demographics representative of the target use case.
• Performance thresholds: What is the maximum acceptable MAE for heart rate (e.g., +/- 3 beats per minute)?

Feature	Off-the-Shelf Generic Model	Custom-Trained rPPG Model
Hardware Specificity	Low (Trained on diverse web data)	High (Trained on the specific target camera)
Use Case Optimization	Low (Assumes general, controlled conditions)	High (Optimized for specific lighting, motion)
Performance Ceiling	Moderate	High (Can achieve production-grade accuracy)
Data Requirement	Uses existing, general datasets	Requires a custom, synchronized data collection
Deployment Flexibility	Limited by model architecture	Tuned for specific edge hardware constraints
Development Timeline	Fast to implement for a PoC	Longer (Requires data collection, training, validation)

5. what are the computational and power constraints for deployment?

Where will the model run? On a powerful cloud server or a resource-constrained microcontroller at the edge? The answer dictates the model architecture.

• Edge Deployment: Requires highly efficient models (e.g., MobileNet, quantized networks) that balance accuracy with low latency and minimal power consumption.
• Cloud/Hybrid Deployment: Allows for larger, more complex models but introduces latency and requires connectivity. The process of model quantization and pruning is a key engineering step for edge AI, as explored in multiple studies on efficient deep learning.

6. What Is the Scope of Deliverables?

A "custom model" is not a single file. A complete delivery package should be clearly defined.

• The Model File: In what format (e.g., TensorFlow Lite, ONNX)?
• Integration Code: Sample code or a software development kit (SDK) demonstrating how to run the model and interpret its output.
• Documentation: Detailed explanation of the model's inputs, outputs, and performance characteristics.
• Retraining Rights: Does the agreement include provisions for model updates or retraining as new data becomes available?

7. what are the data ownership and IP licensing terms?

Data is the most valuable asset in any AI project. Clear terms must be established regarding:

• Data Ownership: Who owns the raw data collected during the project? Who owns the trained model?
• Intellectual Property: The OEM is commissioning work, but the AI vendor is often using its own pre-existing IP. How is this reflected in the licensing?
• Licensing Model: Is it a one-time fee, a per-unit royalty, or a subscription?

Industry Applications

For automotive OEMs, these questions directly map to the requirements for building robust driver monitoring systems that can withstand the harsh in-cabin environment. For smart mirror and IoT device makers, the focus may be on optimizing for specific NIR/IR sensors used for low-light performance. In the clinical space, the fidelity of the ground-truth device and the protocol for data collection are placed under the highest scrutiny.

Current research and evidence

The academic and research communities continue to validate the need for a custom approach. Work by researchers like Gerard de Haan (Eindhoven University of Technology, ongoing) has repeatedly highlighted the sensitivity of rPPG algorithms to subtle hardware and environmental variables. Studies on domain adaptation, such as the 2018 paper by Wang et al., provide a framework for understanding how to adapt models from a source domain (e.g., a public dataset) to a target domain (the OEM's specific hardware). This body of research confirms that achieving high accuracy is not a matter of finding a single "magic" algorithm, but of careful, domain-specific engineering.

The future of custom rPPG modeling

The trend is toward even greater specialization. We are moving from camera-specific models to "environment-and-population-specific" models. Future developments will likely involve federated learning, where models can be trained across a fleet of devices without centralizing raw user data, addressing privacy concerns. Furthermore, the fusion of rPPG with other sensor modalities, like thermal imaging for respiratory rate or radar for body motion, will require even more sophisticated custom models that can interpret multi-modal data streams.

Frequently asked questions

Q: What is the main difference between model "tuning" and a full custom build? A: Model tuning, or transfer learning, involves adapting a pre-existing base model to new data. It's effective when the target camera is similar to the original training cameras. A full custom build is a more extensive process, often involving a new model architecture and a large-scale, dedicated data collection effort, necessary for novel sensors or challenging use cases.

Q: How much data is required to train a custom rPPG model? A: The amount varies widely based on the complexity of the use case. A simple application in a controlled environment might require a few dozen hours of synchronized data. A robust automotive application that must work for diverse drivers in all lighting conditions could require thousands of hours. The key is data diversity, not just volume.

Q. Can one custom model work on a family of related sensors? A. It's possible but requires careful evaluation. If a family of sensors shares the same core imaging characteristics and ISP pipeline, a model trained on data from all of them may generalize well. However, even minor differences in firmware or lens coatings can impact performance, often necessitating at least a final tuning step for each specific model.

Asking these seven questions is the first step toward a successful partnership and a product that uses camera-based sensing to its fullest potential. The process is complex, but for OEMs seeking a true competitive advantage, a custom-trained model is the only way to ensure the reliability and accuracy their customers expect. Circadify's research team specializes in addressing these challenges. To discuss your specific hardware and use case, inquire about a custom build.