UNSOLVED Assistance Needed with RAW-MIPI-IMX462M Camera Integration on NVIDIA AGX Orin

barbanevosa

I am using the RAW-MIPI-IMX462M camera module on an NVIDIA AGX Orin platform for my project. Below are my system details:

$ uname -r
5.15.148-tegra

$ cat /etc/nv_tegra_release 
# R36 (release), REVISION: 4.4, GCID: 41062509, BOARD: generic, EABI: aarch64, DATE: Mon Jun 16 16:07:13 UTC 2025
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia

Previously, I successfully used the MV-MIPI-IMX178M camera with VLC and GStreamer via v4l2 without any issues.

I have recently updated the device tree overlay (*.dtbo) for the new RAW-MIPI-IMX462M, and the camera is detected correctly as shown below:

$ ./mv_mipi_i2c_new.sh -r model -b 9
model is RAW-MIPI-IMX462M

$ v4l2-ctl --device=/dev/video0 --all
Driver Info:
        Driver name      : tegra-video
        Card type        : vi-output, mvcam 9-003b
        Bus info         : platform:tegra-capture-vi:0
        Driver version   : 5.15.148
        Capabilities     : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : tegra-camrtc-ca
        Model            : NVIDIA Tegra Video Input Device
        Serial           : 
        Bus info         : 
        Media version    : 5.15.148
        Hardware revision: 0x00000003 (3)
        Driver version   : 5.15.148
Interface Info:
        ID               : 0x03000017
        Type             : V4L Video
Entity Info:
        ID               : 0x00000015 (21)
        Name             : vi-output, mvcam 9-003b
        Function         : V4L2 I/O
        Pad 0x01000016   : 0: Sink
          Link 0x0200001b: from remote pad 0x1000003 of entity '13e00000.host1x:nvcsi@15a00000-' (Unknown sub-device (0002000a)): Data, Enabled
Priority: 2
Video input : 0 (Camera 0: no power)
Format Video Capture:
        Width/Height      : 1920/1080
        Pixel Format      : 'XY10' (XAVIER 10-bit/16-bit Greyscale)
        Field             : None
        Bytes per Line    : 3840
        Size Image        : 4147200
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             : 

User Controls

                   trigger_mode 0x00981901 (int)    : min=0 max=2 step=1 default=0 value=0 flags=volatile, execute-on-write
                    trigger_src 0x00981902 (int)    : min=0 max=1 step=1 default=1 value=1 flags=volatile, execute-on-write
                    soft_trgone 0x00981903 (button) : value=0 flags=write-only, execute-on-write
                     frame_rate 0x00981904 (int)    : min=0 max=61 step=1 default=60 value=60 flags=volatile, execute-on-write
                          roi_x 0x00981905 (int)    : min=0 max=1576 step=8 default=0 value=0
                          roi_y 0x00981906 (int)    : min=0 max=792 step=4 default=0 value=0

Camera Controls

error 22 getting ext_ctrl Sensor configuration
error 22 getting ext_ctrl Sensor mode I2C packet
error 22 getting ext_ctrl Sensor control I2C packet
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
                                0: 0 (0x0)
                                1: 1 (0x1)
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
                                0: 0 (0x0)
                                1: 1 (0x1)
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0 (1 0x1)
                                0: 1 (0x1)
                                1: 65536 (0x10000)
                                2: 131072 (0x20000)
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][18] flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][36] flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
    override_capture_timeout_ms 0x009a206f (int)    : min=-1 max=2147483647 step=1 default=2500 value=2500
            vi_time_out_disable 0x009a2078 (bool)   : default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only

Image Processing Controls

                     pixel_rate 0x009f0902 (int64)  : min=750000000 max=750000000 step=1 default=750000000 value=750000000 flags=read-only

Currently, I am having difficulty capturing video streams in the XY10 or XY12 formats using VLC, GStreamer, or other players. I can capture raw images with v4l2-ctl that require conversion to a suitable format, but my intended workflow involves capturing video with Python, OpenCV, and the CUDA toolchain.

Could you please assist me in setting up the correct workflow to capture and process video from this camera using these tools?

Thank you for your support.

barbanevosa

@veye_xumm
The camera is detected by the system, but I can't seem to get a video stream from it. I keep running into issues, and I'm not sure what I'm doing wrong.

Could you please help me with the following:

What is the correct way to capture video from this specific camera model?
Are there any common pitfalls or specific settings I should be aware of?
What might I be missing in my current setup?

Any advice, examples, or pointers to relevant documentation would be immensely helpful.

Thank you in advance for your support and expertise.

veye_xumm

@barbanevosa
Sorry, I just got back from the National Day holiday. The images from the RAW-MIPI-IMX462M module are directly output by the sensor and do not support RAW8, only RAW10 and RAW12 data formats. This indeed makes it incompatible with universal players.

veye_xumm

@barbanevosa Actually, if you need to use video stream mode, it is recommended to use our color camera VEYE-MIPI-IMX462, which also supports monochrome image output.
https://wiki.veye.cc/index.php/VEYE-MIPI-IMX462_Data_Sheet

barbanevosa

@veye_xumm
I'm experiencing issues with capturing a still image from my RAW-MIPI-IMX462M camera.

Current image processing pipeline:

v4l2-ctl -d 0 -v width=1920,height=1080,pixelformat=XY12 --stream-mmap --stream-count 1 --stream-to captured_frame.raw

magick -size 1920x1080 -depth 16 -endian LSB gray:captured_frame.raw -depth 8 frame.png

The problem:

First capture after boot produces correct image:
All subsequent captures yield corrupted images:
Camera reboot restores proper functionality.

./mv_mipi_i2c_new.sh -w reboot -b 9

Troubleshooting performed:

Tested both XY10 and XY12 pixel formats
Used different conversion tools (ImageMagick, ffmpeg)
Tried 8-bit and 16-bit grayscale output
Applied pixel bit shifting (both pixel_layer_convert and custom scripts)
Compared camera settings pre/post capture using:

v4l2-ctl -d 0 -L

Read internal camera registers via ./mv_mipi_i2c.sh

No configuration differences were detected between "working" and "failing" camera states.

What could cause this single-good-frame behavior, and how can it be resolved?