When Does a Vision-Guided Cobot Need a Depth Camera Instead of 2D?
    Back to Blog
    Robotics

    When Does a Vision-Guided Cobot Need a Depth Camera Instead of 2D?

    2026-11-13T06:01:15.64577+00:005 min readBy smert.ai Team

    When does a vision-guided cobot need a depth camera instead of a 2D camera?

    A vision-guided cobot needs depth sensing when the task requires spatial information that a calibrated 2D approach cannot reliably establish from known geometry and controlled presentation. Do not choose depth merely because the robot moves in three dimensions. First test whether the missing information affects the task, then compare camera options against the same acceptance criteria.

    What information is actually missing?

    Start with the decision the robot must make, not the camera catalogue. Does it need an item's location on a controlled surface, its orientation relative to a fixture, or a height that changes between presentations? Those are different measurement requirements.

    For robotic vision systems, the useful distinction is between information established by the process and information that must be measured each time. A fixture may establish where a part sits. Known part geometry may support pose estimation. If neither establishes the required height or orientation, that uncertainty becomes a reason to evaluate depth sensing.

    Write the requirement as a testable statement: “The system must estimate this target's position and orientation within our task-specific tolerance across these allowed presentations.” Avoid “the camera must understand the scene,” which gives an evaluator no clear pass condition.

    Can a 2D camera estimate a three-dimensional pose?

    Yes, when the necessary known geometry and observations are available. A 2D image does not automatically restrict the output to a two-dimensional location.

    The supplied OpenCV calibration tutorial describes estimating camera intrinsics and distortion from chessboard observations; it also identifies solvePnP as a way to estimate the pose of known geometry. This supports evaluating a calibrated 2D approach before assuming that a depth camera is necessary.

    The limitation is important: known geometry is an input, not a result the method can simply invent. Ask whether the relevant geometry is actually known and whether the observations needed to estimate its pose remain available throughout the intended task.

    For example, consider a proposed inspection-to-pick process using a consistently presented part. The first evaluation question should be whether the calibrated image and known part geometry meet the position requirement. This is a test scenario, not evidence that a particular implementation will succeed.

    When is depth worth testing instead?

    Depth becomes a meaningful candidate when the unresolved measurement is spatial and materially changes the required target pose. Consider three questions:

    • Does height vary outside what the process establishes? If the target height is not fixed or otherwise determined, test how each approach estimates it.
    • Does the known-geometry assumption hold? If parts or presentations change, do not silently reuse an assumption validated for a different case.
    • Can presentation be constrained instead? Compare measuring the variation with changing the fixture or process so that less variation needs measuring.

    These are reasons to run a comparison, not proof that a depth camera will solve the problem. The supplied evidence contains no comparative camera benchmarks, material-specific performance results or sensor specifications. It therefore cannot establish which depth technology, model or configuration will meet your requirement.

    The practical trade-off is between controlling the presentation and measuring more of its variation. Neither route should be selected without testing the actual acceptance conditions.

    Does depth remove the need for calibration?

    No. Choosing a camera and relating its measurements to robot coordinates are separate decisions.

    MoveIt's calibration explanation describes extrinsic calibration as relating the camera pose to a machine coordinate frame. For an eye-in-hand arrangement, camera-to-target and base-to-end-effector pose pairs support calibration.

    Consequently, a comparison should evaluate the resulting target estimate in the robot's coordinate frame, not merely how convincing an image or depth display looks. Detecting a calibration board is not, by itself, an evaluation of calibration quality.

    Neither a camera picture nor an AI interpretation is a validated safe motion command. Camera selection does not replace the separate engineering and validation needed before motion is released.

    What should the evaluation checklist include?

    Use the same task definition for both candidates:

    • Required output: Specify which position and orientation components the task needs.
    • Allowed variation: List the part geometries, heights, orientations and presentations the process permits.
    • Known inputs: Document what fixtures, part models or other process constraints establish.
    • Acceptance limits: Set task-specific tolerances before comparing results; do not borrow an unsupported accuracy figure.
    • Calibration evaluation: Check estimates against reference positions across the intended working region, not only at one convenient location.
    • Unresolved cases: Record missing estimates and unacceptable estimates separately. Define what should happen when the required information is unavailable.
    • Coordinate conversion: Evaluate the camera-to-machine relationship alongside the camera measurement.
    • Decision record: State why the selected approach meets the requirement and which conditions remain outside its evaluated scope.

    For a SMERT Cobot project, discuss these requirements with SMERT as an OEM and integrator through its robotics service. Ask for a task-specific evaluation plan rather than treating a camera category as a performance promise.

    Three useful follow-up questions

    1. Which unknown actually prevents using 2D? Name the missing height, orientation or geometry information, and explain how it changes the target pose.
    2. Could a presentation change remove that unknown? Compare a fixture or process adjustment with adding depth measurement, including how each option would be evaluated.
    3. What evidence would justify changing the camera choice? Agree in advance which failed acceptance tests would trigger a depth trial, a revised 2D setup or a narrower task scope.

    Related: Cobot integration

    Transform Your Business with SMERT Robotics

    Ready to experience the benefits of hospitality automation? Our team is ready to help you get started.

    Explore Solutions