Other
PyTorch
android

CVT: Optimized for Qualcomm Devices

Cross-View Transformer generates real-time bird's-eye view maps from multiple vehicle cameras for autonomous driving.

This is based on the implementation of CVT found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.51, ONNX Runtime 1.30.0 Download
ONNX w8a16_mixed_int16 Universal QAIRT 2.51, ONNX Runtime 1.30.0 Download
QNN_DLC float Universal QAIRT 2.51 Download
QNN_DLC w8a16_mixed_int16 Universal QAIRT 2.51 Download
TFLITE float Universal QAIRT 2.51 Download

For more device-specific assets and performance metrics, visit CVT on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for CVT on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.driver_assistance

Model Stats:

  • Inference latency: RealTime
  • Input resolution: 1x6x3x224x480
  • Model checkpoint: vehicles_50k.pt
  • Model size (float): 5.18 MB
  • Number of parameters: 1.33M

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
CVT ONNX float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 144.002 ms 3 - 2050 MB NPU
CVT ONNX float Snapdragon® 8 Elite For Galaxy Mobile 162.689 ms 4 - 2017 MB NPU
CVT ONNX float Snapdragon® X2 Elite 147.981 ms 8 - 8 MB NPU
CVT ONNX float Snapdragon® X Elite 243.374 ms 17 - 17 MB NPU
CVT ONNX float Snapdragon® 8 Gen 3 Mobile 214.71 ms 8 - 2497 MB NPU
CVT ONNX float Snapdragon® 8 Gen 1 Mobile 332.079 ms 8 - 2465 MB NPU
CVT ONNX float Qualcomm® Dragonwing™ IQ-8275 227.669 ms 7 - 12 MB NPU
CVT ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 282.719 ms 8 - 12 MB NPU
CVT ONNX float Qualcomm® QCS8450 332.079 ms 8 - 2465 MB NPU
CVT ONNX float Qualcomm® Dragonwing™ IQ-9075 249.665 ms 7 - 11 MB NPU
CVT ONNX float Qualcomm® Dragonwing™ IQ-X7181 243.374 ms 17 - 17 MB NPU
CVT ONNX float Qualcomm® Dragonwing™ Q-8750 162.689 ms 4 - 2017 MB NPU
CVT ONNX float Snapdragon® 8 Elite Extreme Gen 6 Mobile 127.993 ms 8 - 2095 MB NPU
CVT ONNX float Snapdragon® 8 Elite Gen 6 Mobile 183.797 ms 8 - 2146 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 190.066 ms 0 - 2179 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® 8 Elite For Galaxy Mobile 185.217 ms 1 - 2090 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® X2 Elite 178.51 ms 4 - 4 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® X Elite 277.608 ms 22 - 22 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® 8 Gen 3 Mobile 236.557 ms 4 - 2552 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® 8 Gen 1 Mobile 326.637 ms 4 - 2405 MB NPU
CVT ONNX w8a16_mixed_int16 Qualcomm® Dragonwing™ IQ-8275 253.052 ms 4 - 8 MB NPU
CVT ONNX w8a16_mixed_int16 Qualcomm® Dragonwing™ QCS8550 (Proxy) 311.361 ms 0 - 29 MB NPU
CVT ONNX w8a16_mixed_int16 Qualcomm® QCS8450 326.637 ms 4 - 2405 MB NPU
CVT ONNX w8a16_mixed_int16 Qualcomm® Dragonwing™ IQ-9075 270.044 ms 4 - 7 MB NPU
CVT ONNX w8a16_mixed_int16 Qualcomm® Dragonwing™ IQ-X7181 277.608 ms 22 - 22 MB NPU
CVT ONNX w8a16_mixed_int16 Qualcomm® Dragonwing™ Q-7790 1777.069 ms 66 - 821 MB NPU
CVT ONNX w8a16_mixed_int16 Qualcomm® Dragonwing™ Q-8750 185.217 ms 1 - 2090 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® 7 Gen 4 Mobile 1777.069 ms 66 - 821 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® 8 Elite Extreme Gen 6 Mobile 145.96 ms 4 - 2346 MB NPU
CVT ONNX w8a16_mixed_int16 Snapdragon® 8 Elite Gen 6 Mobile 211.641 ms 4 - 2299 MB NPU
CVT QNN_DLC float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 143.684 ms 7 - 2059 MB NPU
CVT QNN_DLC float Snapdragon® 8 Elite For Galaxy Mobile 162.018 ms 7 - 2006 MB NPU
CVT QNN_DLC float Snapdragon® X2 Elite 147.6 ms 7 - 7 MB NPU
CVT QNN_DLC float Snapdragon® X Elite 242.425 ms 7 - 7 MB NPU
CVT QNN_DLC float Snapdragon® 8 Gen 3 Mobile 217.997 ms 5 - 2461 MB NPU
CVT QNN_DLC float Snapdragon® 8 Gen 1 Mobile 331.884 ms 7 - 2456 MB NPU
CVT QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 227.213 ms 7 - 18 MB NPU
CVT QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 286.016 ms 8 - 1045 MB NPU
CVT QNN_DLC float Qualcomm® QCS8450 331.884 ms 7 - 2456 MB NPU
CVT QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 248.183 ms 7 - 17 MB NPU
CVT QNN_DLC float Qualcomm® Dragonwing™ IQ-X7181 242.425 ms 7 - 7 MB NPU
CVT QNN_DLC float Qualcomm® Dragonwing™ Q-8750 162.018 ms 7 - 2006 MB NPU
CVT QNN_DLC float Qualcomm® SA8295P 287.47 ms 0 - 2099 MB NPU
CVT QNN_DLC float Snapdragon® 8 Elite Extreme Gen 6 Mobile 127.966 ms 7 - 2082 MB NPU
CVT QNN_DLC float Snapdragon® 8 Elite Gen 6 Mobile 183.924 ms 8 - 2083 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 170.866 ms 4 - 2203 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Snapdragon® 8 Elite For Galaxy Mobile 189.159 ms 4 - 2114 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Snapdragon® X2 Elite 178.042 ms 4 - 4 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Snapdragon® X Elite 309.319 ms 4 - 4 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Snapdragon® 8 Gen 3 Mobile 243.781 ms 5 - 2486 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Qualcomm® Dragonwing™ IQ-8275 290.679 ms 4 - 10 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Qualcomm® Dragonwing™ QCS8550 (Proxy) 348.706 ms 4 - 8 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Qualcomm® Dragonwing™ IQ-9075 303.09 ms 4 - 10 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Qualcomm® Dragonwing™ IQ-X7181 309.319 ms 4 - 4 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Qualcomm® Dragonwing™ Q-8750 189.159 ms 4 - 2114 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Snapdragon® 8 Elite Extreme Gen 6 Mobile 150.602 ms 4 - 2438 MB NPU
CVT QNN_DLC w8a16_mixed_int16 Snapdragon® 8 Elite Gen 6 Mobile 214.931 ms 4 - 2205 MB NPU
CVT TFLITE float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 145.861 ms 1 - 2036 MB NPU
CVT TFLITE float Snapdragon® 8 Elite For Galaxy Mobile 162.317 ms 1 - 2008 MB NPU
CVT TFLITE float Snapdragon® 8 Gen 3 Mobile 214.275 ms 1 - 2502 MB NPU
CVT TFLITE float Snapdragon® 8 Gen 1 Mobile 329.95 ms 0 - 2466 MB NPU
CVT TFLITE float Qualcomm® Dragonwing™ IQ-8275 228.287 ms 0 - 34 MB NPU
CVT TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 279.868 ms 0 - 4 MB NPU
CVT TFLITE float Qualcomm® SA8650P 571.046 ms 48 - 61 MB CPU
CVT TFLITE float Qualcomm® SA8255P 571.046 ms 48 - 61 MB CPU
CVT TFLITE float Qualcomm® QCS8450 329.95 ms 0 - 2466 MB NPU
CVT TFLITE float Qualcomm® Dragonwing™ IQ-9075 251.171 ms 0 - 35 MB NPU
CVT TFLITE float Qualcomm® Dragonwing™ Q-8750 162.317 ms 1 - 2008 MB NPU
CVT TFLITE float Qualcomm® SA7255P 826.236 ms 48 - 57 MB CPU
CVT TFLITE float Qualcomm® SA8295P 289.018 ms 1 - 2097 MB NPU
CVT TFLITE float Snapdragon® 8 Elite Extreme Gen 6 Mobile 128.307 ms 0 - 2081 MB NPU
CVT TFLITE float Snapdragon® 8 Elite Gen 6 Mobile 128.595 ms 0 - 2083 MB NPU

License

  • The license for the original implementation of CVT can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/CVT