Orbbec Cameras Integrated with NVIDIA Isaac Robotics Platform        

One place where technology and integration and partners advances lies in the vision and robotics area. This news concerns Orbbec 3D vision systems integrating with NVIDIA Isaac Perceptor robotics platforms.

In brief: Gemini 330 cameras with built-in depth processing deliver high-precision Depth+RGB vision for NVIDIA  Isaac Perceptor AI-based perception workflow for autonomous mobile robots (AMRs) in indoor and outdoor environments.

Orbbec, an industry leader dedicated to 3D vision systems, announced its Gemini 330 series Stereo Vision 3D cameras are now integrated with NVIDIA Isaac Perceptor, a reference workflow for autonomous mobile robots (AMRs) built on GPU-accelerated Isaac ROS.

These cameras enhance depth quality and provide longer-range sensing in varied lighting conditions, which lets Isaac Perceptor – whose general availability was announced by NVIDIA today at COMPUTEX – output 3D reconstruction and obstacle cost maps of any unstructured environment.

The Gemini 335/335L/336/336L cameras operate in both passive and active laser-illuminated modes to ensure high-quality depth and RGB data output even in challenging lighting conditions. The depth algorithms are processed in the camera by Orbbec’s latest depth engine ASIC and thus eliminates the burden on the NVIDIA Jetson Orin module-based compute for such operations. The cameras include internal IMU and temperature sensors and have a working range of 0.2-10 meters, global shutter image sensors, wide field-of-view lenses, high frame rates, low latency and precise multi-camera synchronization.

Orbbec also announces the Gemini 336/336L variants for improved performance in indoor environments by adding NIR bandpass filters. This reduces the potential of “holes” in a depth map due to glare from shiny floors and other reflective surfaces and “ghost” images from repetitive patterns in the environment.

In addition to AMRs, the Gemini 330 series cameras are well suited for robot arm applications that utilize AI vision for bin-picking, palletization, scanning and sorting applications, especially where reduction in glare and resulting holes from glossy surfaces are important.

Mitsubishi Electric Corporation Leads Series B Investment in Realtime Robotics

Investments seldom interest me. This one in the robotics area should be noticed when we consider the usual lifecycle path of a start up technology company.

Realtime Robotics, the leader in collision-free autonomous motion planning for industrial robots, today announced that it has secured a strategic investment from Mitsubishi Electric Corporation. This is the lead investment in Realtime Robotics’ recently opened Series B round. Mitsubishi Electric was also a participant in the Series A round, and will be adding a senior representative to Realtime’s Board of Directors.

Realtime Robotics’ unique and innovative collision-free path planning technology provides solutions across the lifecycle of robotic workcells. In iterative design stages, the award-winning multirobot optimization software rapidly generates and evaluates hundreds of thousands of possible solutions to identify the shortest cycle time. Deployment and production are further simplified by runtime control, enabling multiple robots to work closer together, while simultaneously reacting to dynamic changes. Finally, when the workcell needs to be retooled, the complex robot control is effortlessly reprogrammed for optimal cycle time from the first iteration. 

Why is Mitsubishi interested?

By increasing its stake, Mitsubishi Electric plans to further integrate Realtime’s motion planning technology into 3D simulators and other software to optimize manufacturing through the power of digital twins. Later, Mitsubishi Electric expects to incorporate Realtime’s technology into factory automation (FA) control system devices, such as programmable logic controllers (PLCs), servo motors and computer numerical controllers (CNCs), to ensure uninterrupted plant operations by responding to needs for expanded automation capabilities, streamlined plant operations for improved efficiency, and fast responses to unexpected events.

Automate Report 6: Technologies

This final report from the Automate 2024 show encompasses some unique technology applications along with other technologies. Featured are Vention, Zebra, LexxPluss, Orbbec, Ericsson and Dassault.


I have written about Vention a few times. I first met them as an innovative component designer. They told me they’ve recently combined five business. Their unique take on technology is that you can Design and Build your machine from their CAD and components on the Web. You go to their CAD and find amenu of components they offer. You design your machine and price BOM, all on Web. It’s sort of like assembling Legos. You can check it out—the CAD part is free to use. You can also add code including Python and C, then simulate machine, and purchase system. You deploy the system with their controller, which can track analytics. Service is prompt and efficient.

Zebra Technologies Advancing the Connected Factory

Zebra’s latest fixed industrial scanner, the Zebra FS42 is designed for resource-intensive tasks like deep learning-based optical character recognition (OCR) applications and high-throughput scanning requiring speed. Its increased onboard memory and neural processing unit (NPU) enable faster scanning and more complex artificial intelligence (AI)-based deep learning machine vision applications. 

Zebra’s Aurora Software Suite is ideal for addressing the track-and-trace and vision inspection needs of users at all experience levels. It supports barcode reading and additional inspection tasks like OCR, dimensioning, and defect detection. The FS42 features Zebra’sAurora Focus which makes it easy to set up, deploy, and run the device, eliminating the need for other tools and reducing training time. 

Zebra’s new 3S Series 3D sensors will also be introduced at Automate. Using patented parallel structured light, the industrial-grade 3S series enables 3D sensing for static and moving items – including those that are high-gloss, translucent, or transparent – with sub-millimeter resolution and accuracy. 

Zebra’s 3S series captures various sizes and is “plug-and-play” ready with a Power over Ethernet (PoE) connection. They will be bundled with Zebra’s Aurora Design Assistant or Aurora Vision Studio which speeds up development and brings 3D vision applications online quicker. 

The 3S40 sensor is designed to scan static objects, integrating them into manufacturing operations to resolve inspection and dimensioning applications with a scanning range of up to 7 t./2.1 meters. The 3S80 sensor is built to capture dynamic scenes and create 3D point color clouds in real time, expanding the range of advanced automation applications. It has a scan range of up to 10 ft./3 meters. 

Zebra Symmetry Fulfillment comprises a comprehensive solution that helps workers and robots be more productive, optimizing the path of a picker with a team of robots for a directed workflow.  Zebra Symmetry Fulfillment uses the power of autonomous mobile robots (AMRs), wearable technologies, software, and analytics to maintain high-quality standards while improving performance and reducing costs per unit.

LexxPluss launches scalable, interoperable mobile conveying

The Japanese intralogistics and mobile robotics company featured the Lexx500 autonomous mobile robot, the LexxTug towing interface, the LexxFleet management system and its IoT LexxHub, which facilitates the collective automation of existing systems with Lexx500 AMRs for enhanced safety and interoperability.

Lexx500 next-generation automated transfer robots with multi-sensor safety features can be deployed from just a single unit and easily increased to a multi-AMR fleet, providing market-leading scalability. The Lexx500 requires no fixed equipment and can automatically transfer heavy loads/objects up to 300kg (660 lbs) with minor modifications, or 500kg (1100 lbs) as is with LexxTug attached.

The LexxTug towing interface allows manufacturers to use general-purpose carts such as cage carts and 6-wheel carts without modification to automatically transfer loads of up to 500kg (1100 lbs). It has an automatic detachment function to enable it to fit into existing operations without major changes.

The LexxFleet management system delivers task management, group control management of multiple units, and system integration. The fleet management system provides users with Lexx500 transport status management, map display, multiple task execution/reservation, RESTful API integration, and remote emergency stop function for the safe, efficient management of AMR fleets.

LexxHub is an IoT solution that enables collective automation of existing systems by allowing existing equipment to connect to the network and work with a fleet of Lexx500 AMRs, creating an autonomous worksite through equipment cooperation. Collaborative tasks such as calling an elevator, production equipment lines, or fire prevention functions such as shutters and alarms can be performed in time with AMRs to enhance productivity and safety.

Orbbec Gemini 330 Series of Stereo Vision 3D Cameras

Orbbec, an industry leader dedicated to 3D vision systems, introduced its Gemini 330 series, featuring the Gemini 335 and Gemini 335L Stereo Vision 3D cameras. Engineered as universal solutions for diverse 3D application scenarios, these cameras boast Orbbec’s latest depth engine chip MX6800, a purpose-built ASIC and patented hybrid stereo imaging technology that ensures stable, high-quality depth data output even in challenging lighting conditions.

Key Features of the Gemini 335 and 335L:

  • Stable, high-quality depth images under various lighting conditions, from direct sunlight to darkroom.
  • Diagonal field of view over 100°, working range beyond 0.2 – 10 meters, global shutter image sensors, high frame rate, low latency, comprehensive and precise sensor synchronization ideal for mobile robotics applications
  • Compact form factor, low power consumption for flexible integration into customer product designs

Ericsson and Dassault

I began this section with an innovative way to design and price a machine. I end with another cool innovation.

Ericsson is known for wireless, 5G cellular private networks and the like. It has partnered with Dassault Systemmes to construct a Digital Twin with Delmia of the network coverage in a plant. They do a LiDar scan of plant to map the plant and its potential steel barriers to signals. They find antennas in the plant with strength of signals and generate a visual map of the plant with colors that show strength of signal. Managers can make better decisions about where they need coverage in a weak spot and do things such as move equipment or racks to provide adequate wireless coverage. The system is also useful for large events.

Automate Report 5: Robots

This report from Automate 2024 consolidates robotic-oriented news from Doosan Robotics, Kassow Robots, InOrbit CoPilot, and Hirebotics. Collaborative Robots, or Cobots, are now just about everywhere. The software for these is innovative, often no-code, and easy-to-use. There is consolidation of the market. It will be interesting to see what next year brings.

Doosan Robotics to Unveil New Cobot

Doosan Robotics unveiled its latest industrial cobot which boasts a 30 kg payload. It also wins the award for the most marketing hype of any press release I received. Parsing through the release, here are a few tidbits.

The P3020 offers a payload of 30kg (60lbs) and reach of 2,030mm (80in), bringing the ability to palletize from floor to 2m high (stacking up to 10 layers of boxes approximately 8 inches tall) using its simple fixed base without a lift.

Additional key features of the P-SERIES cobot lineup include lower power consumption compared to similar payload cobots by applying its built-in gravity compensation mechanism, inherent wrist-singularity free, and a 5 degree-of-freedom movement with the 4th axis removed and 6th axis speed increased to 360 degrees/second. Doosan Robotics’ continued priority on optimal safety across its solutions is also fully present in the new P3020. This includes achieving the highest PL (e) and Cat 4 safety ratings to ensure both a max-powered and max-safety experience for users. 

Dart-Suite an advanced robot ecosystem designed to redefine the way customers utilize Doosan cobots. The scalable platform provides “unlimited” programmable motions, while ensuring accessibility for all.

Kassow Robots launches the world’s first 7-axis cobot series with a controller integrated into the base

With its Edge Edition, Kassow Robots has launched the world’s first 7-axis cobots in which the controller is integrated into the base of the robot. All five of the 7-axis cobots are now available in two versions: the classic variant with a separate controller, and the Edge Edition. Kristian Kassow and his engineering team have succeeded in miniaturizing the controller so that it now only occupies around 10% of the volume of the external controller. The footprint of the Edge edition’s base is compact, measuring only 160 by 200 millimeters. These cobot models are powered by direct current, meaning that they can be directly connected to any DC power supply, for example a battery of mobile robots. There are also easily accessible 60 IO connections. With these new models, the Copenhagen-based cobot manufacturer is making it easier for industrial companies to automate their operations. At the same time, their lightweight robots open up new opportunities for mobile cobot AMR and cobot AGV applications and various space-saving solutions.

InOrbit Unveils RobOps Copilot for AI-Powered Robot Optimization

InOrbit.AI, the leading provider of robot operations (RobOps) solutions, introduced InOrbit RobOps Copilot. This product harnesses the power of artificial intelligence to turn robot operations data into actionable insights in order to optimize autonomous robot fleets.

Leveraging the latest large language models (LLMs), RobOps Copilot enables users to ask questions in their preferred language, get detailed explanations, refine their analysis, and arrive at key optimization decisions, seamlessly transitioning between their favorite messaging platform for casual queries and a fully integrated experience with chat-driven dashboard explorations.

As with the rest of the InOrbit platform, RobOps Copilot works with mixed, distributed robot fleets. InOrbit continues to expand the InOrbit Connect ecosystem, including the ability to define, execute, and analyze missions that support integrations with Warehouse Management Systems, AMRs from different vendors, and Goods-to-Person workflows. One recent addition is support for a novel Automated Storage and Retrieval System developed by Instock, which is now in use at the InOrbit Robot Space in Silicon Valley and will be part of InOrbit’s live remote demonstrations presented at Automate.

Beacon-Powered Hirebotics Cobot Welder

Hirebotics launched support for the new Miller Auto DeltaWeld with their Cobot Welder. The Miller Auto DeltaWeld System is a MIG/Flux-cored, pulsed-capable power source designed for cobots. 

Unlike other cobot solutions, Hirebotics Beacon-powered Cobot Welder lets the user control everything in one place—a simple smartphone/tablet interface. This makes the process easier and faster compared to systems where welders must go back and forth between the teach pendant and the power source. 

Beacon is cloud-based, allowing total remote and local cobot control.

Automate Report 4: Festo Introduces Several New Products

Festo is an innovative company that continuously surprises me with its creativity. This year at Hannover, for example, they displayed a “flock” of bees. At Automate they showed new electric actuators, an I/O platform, and an AI worker collaboration tool.

New Generation of Electric Actuators for Linear Applications

Festo introduced its newest generation of Cartesian-handling-system mechanical axes. The ELGD-TB, tooth belt actuators, and ELGD-BS, ball screw actuators, are based on an innovative Festo design. 

  • Suited to most electromechanical handling tasks, the ELGD series brings specific benefits to applications, such as:
  • Cantilever systems and pick-and-place solutions for small parts handling, where short cycle times, high precision, and repeatability are key
  • Handling systems for top loaders, which benefit from the ELGD’s attractive price/performance ratio, high travel speed, and long stroke lengths
  • Automation of 3D printing, additive manufacturing, dosing, gluing, and picking and placing are enhanced with the ELGD’s dynamic, virtually vibration-free movement.  

Remote and Decentralized I/O Platform

Festo offers both the CPX-AP-A distributed I/O and CPX-AP-I decentralized I/O. Whether physically attached on-terminal or connected via cable, all components such as PLCs, valves, motors, drives, and I/O appear to the programmer to be incorporated within one smart terminal under a single IP address. Having the entire distributed and decentralized I/O topology under a single IP address significantly reduces hardware and installation costs while lowering system complexity. Since many PLCs come with a limited number of IP addresses, the Festo solution means more capabilities are possible per PLC. Festo is a leader in adopting and applying IO-Link, and customers will find IO-Link capabilities integral to the new I/O automation platform. This I/O platform is ideal for effective conveyor control.

AI worker collaboration training package

At the show, Festo demonstrates a UR3e cobot equipped with an AI-powered camera capable of gesture recognition. This setup enables object tracking, allowing the cobot to pick up items and then hand them to booth visitors. The cobot will be handing out individually wrapped hard candy. The Festo Didactic AI/Machine Learning Worker Collaboration package provides learners with the opportunity to explore neural networks, machine learning, and advanced gripping systems. Festo Didactic incorporated into this demo its 360-degree robotic safety awareness package developed in partnership with SICK.

Averroes.ai Brings Apple Vision Pro to Manufacturing for Inspection

Someone else is going to transform the industry. OK, I have worn many Augmented Reality glasses in demos at trade shows. They are cool, but it still looked like a solution searching for a problem. Meanwhile, the Apple Vision Pro supports both AR and Virtual Reality (VR). Everyone is talking about how cool the VR part is for immersive movies. No one has been talking about the AR capabilities.

Until, now.

Averroes.ai has introduced an what they call an “innovative approach to manufacturing that is set to transform the industry.” They are combining the ability to talk to machines sucking in data and visualizing it in real time using the Apple Vision Pro in AR mode.

  • Averroes.ai uses machine learning and AI to detect, classify, and segment defects, and carry out predictive maintenance.
  • Manufacturers can even spot defects along production lines without direct observation, all without the need to be seated in a back office.
  • This integration facilitates instant access to production data and charts, empowering on-the-spot monitoring, defect analysis, statistical insights, and yield optimization, all with unparalleled ease and accuracy, right on the factory floor.

Averroes.ai’s innovative approach is rooted in being data agnostic, a concept that ensures seamless integration of various data types. Whether it’s images, tabular data, or time series, their system effortlessly accommodates them all.

Business reasons. According to recent reports, the global AI Industrial Vision Inspection Service market is valued at US$ 332 million in 2022. What’s more staggering is its projected growth trajectory. By 2029, this market is expected to skyrocket to a value of US$ 1156.2 million, boasting a compound annual growth rate of 19.7%.

Tareq, CEO of Averroes.ai, told me when I asked about walking through a manufacturing plant wearing one of these things, “The Apple Vision Pro uses augmented reality (AR) to enhance, not hinder, your interaction with the environment. Its AR display is designed to subtly complement your real-world view. Whether it’s showing data labels, instructions, or visual enhancements, everything is carefully designed to be unobtrusive, blending seamlessly into your surroundings.”

Second, when will it actually be available? “Averroes is an AI platform that now allows manufacturers to automatically generate visual inspection APIs from models, including anomaly detection, using a no-code approach. With minimal data and no need for extensive support, manufacturers can develop these models efficiently, utilizing the Apple Vision Pro as their imaging system. The platform is not open to the public and is available exclusively to enterprise clients with a subscription.”

Why? There are other, cheaper solutions. “There has always been a demand for inspecting products in environments where items vary and do not conform to a uniform production line. The Apple Vision Pro and Averroes platform are well-suited to address this need. The Apple Vision Pro offers a cost-effective alternative to traditional fixed-installation industrial cameras and delivers superior image quality and resolution, which is crucial for inspecting products in varying lighting conditions.”

“Although Averroes can also integrate with other, less expensive AR sets, the development kit that comes with the Apple Vision Pro is among the best available, making it an excellent choice for manufacturers tasked with inspecting diverse products in non-standard settings.”

Follow this blog

Get a weekly email of all new posts.