Publications | Holly Dinkel

2024

Image-Based Pose Estimation of Sub-Centimeter Industrial Parts for Automated Assembly

Hameed Abdul-Rashid, Yangfei Dai, Holly Dinkel, Minh Quang Ta, Tan Chen, Junyi Geng, and Timothy Bretl

In CoRL Workshop on Learning Robotic Assembly of Industrial and Everyday Objects, Nov 2024

Abstract PDF Code

This work adapts and integrates existing machine vision techniques to estimate the 6DoF pose of sub-centimeter parts for high-mix, low-volume assembly lines, focusing on the challenges of accurate positioning in real-world scenarios. In this system, the 3D models of each part are input to a BlenderProc2 rendering engine to generate a physically- and photometrically-realistic synthetic image dataset. Synthetic images are used to train a Mask R-CNN model for segmenting individual part instances in a scene, with automatically-generated instance mask labels, eliminating the need for manual labeling. Instance segmentation enables part selection for assembly when multiple parts are present. Additionally, a PVNet model is trained on cropped images of each part instance to estimate their positions and orientations. An additional pose refinement step adjusts PVNet pose estimates by aligning the orientation to the nearest physically-stable configuration on a planar surface and refining the translation using calibrated object-to-camera distances from the workspace. To evaluate robustness, noise is injected into the keypoint detection stage of the PVNet model in an ablation study to assess the impact of sensor noise on pose estimation. Real robot pick-and-place experiments demonstrate the system performance.
Business Innovation in Commercial Space: Culture and Trends in Earth Observation

Giulia Cambone, Holly Dinkel, Luca Ferrone, KangSan Kim, Shinsuke Kito, and Chawalwat Martkamjan

In IAF International Astronautical Congress, Oct 2024

Abstract PDF Code

This work studies technological, business, and cultural development practices in the commercial Earth Observation (EO) sector. Increased access to space combined with growing demands for climate and natural resource monitoring, situational awareness, on-demand analytics, and sovereign data independence motivate the expansion of EO. The financial performance and culture of five public EO companies–iQPS, GOMSpace, Kleos Space, Planet Labs, and Satellogic–are analyzed. Time-series econometrics modeling indicates trends in how financial resources impact earnings and revenue, and the models are used to forecast future earnings and revenue with a high goodness-of-fit. A time-series DuPont analysis investigates how companies operate through conversion of different financial instruments, and corresponding corporate milestones contextualize these financial transactions. Informational interviews of four EO company executives provide clues about company culture and decision-making strategies. Recommendations tailored to the unique challenges and opportunities of the global EO sector are provided to help government partners, investors, and program managers estimate business performance given financial and cultural data.
AstrobeeCD: Change Detection in Microgravity with Free-Flying Robots

Holly Dinkel, Julia Di, Jamie Santos, Keenan Albee, Paulo V.K. Borges, Marina Moreira, Ryan Soussan, Oleg Alexandrov, Brian Coltin, and Trey Smith

Acta Astronautica, Oct 2024

Abstract PDF Code Video

This work presents AstrobeeCD, a system for 3D scene change detection toward near-real-time environmental awareness of space outposts using the Astrobee free-flying robot in microgravity. Assistive free-flyer robots autonomously caring for future crewed space habitats must be able to detect day-to-day interior changes to track inventory, detect and diagnose faults, and monitor the outpost status. A set of image and depth data from one time step is used to reconstruct a 3D model of the environment. The 3D model is used as the basis for comparison for free-flyer environment surveys at future time steps, where an image-based change detection algorithm identifies inconsistencies against the 3D model. Change detection is demonstrated using real image and pose data collected by an Astrobee robot in a test environment on Earth at NASA Ames Research Center and from microgravity aboard the International Space Station. Change detection computation time and performance are quantitatively evaluated on the test data captured on Earth, and it identifies scene changes more quickly than a point cloud clustering-based algorithm applied to data from the same surveys.
KnotDLO: Toward Interpretable Knot Tying

Holly Dinkel, Raghavendra Navaratna, Jingyi Xiang, Brian Coltin, Trey Smith, and Timothy Bretl

In IEEE ICRA Workshop on 3D Visual Representations for Manipulation, May 2024

Abstract PDF Poster Video

This work presents KnotDLO, a method for one-handed knot manipulation to transform a Deformable Linear Object (DLO) into knot states. Grasp and target waypoints for future DLO states are planned from the current DLO shape. Grasp poses are computed from indexing the tracked piecewise linear curve representing the DLO state based on the current curve shape and are piecewise continuous. Intermediate waypoints are computed from the geometry of the current DLO state and the desired next state. The perception and manipulation system is robust to occlusion, repeatable for varying rope initial configurations, interpretable for generating motion policies, and requires no human demonstrations or training. The system decouples visual reasoning from control. In 16 trials of knot tying, KnotDLO achieves a 50% success rate in tying an overhand knot from previously unseen configurations.
TrackDLO: Tracking Deformable Linear Objects Under Occlusion with Motion Coherence

Jingyi Xiang, Holly Dinkel, Harry Zhao, Naixiang Gao, Brian Coltin, Trey Smith, and Timothy Bretl

IEEE ICRA Workshop on Representing and Manipulating Deformable Objects, May 2024

Best Abstract Finalist 🏆

Abstract PDF Code Poster Video

The TrackDLO algorithm estimates the shape of a Deformable Linear Object (DLO) under occlusion from a sequence of RGB-D images. TrackDLO is vision-only and runs in real-time. It requires no external state information from physics modeling, simulation, visual markers, or contact as input. The algorithm improves on previous approaches by addressing three common scenarios which cause tracking failure: tip occlusion, mid-section occlusion, and self-occlusion. This is achieved through the application of Motion Coherence Theory to impute the spatial velocity of occluded nodes, the use of the topological geodesic distance to track self-occluding DLOs, and the introduction of a non-Gaussian kernel that only penalizes lower-order spatial displacement derivatives to reflect DLO physics. Improved real-time DLO tracking under mid-section occlusion, tip occlusion, and self-occlusion is demonstrated experimentally. The source code and demonstration data are publicly released.
Unsupervised Change Detection for Space Habitats Using 3D Point Clouds

Jamie Santos, Holly Dinkel, Julia Di, Paulo V.K. Borges, Marina Moreira, Oleg Alexandrov, Brian Coltin, and Trey Smith

In AIAA SciTech Forum, Jan 2024

Abstract PDF Code Video

This work presents an algorithm for scene change detection from point clouds to enable autonomous robotic caretaking in future space habitats. Autonomous robotic systems will help maintain future deep-space habitats, such as the Gateway space station, which will be uncrewed for extended periods. Existing scene analysis software used on the International Space Station (ISS) relies on manually-labeled images for detecting changes. In contrast, the algorithm presented in this work uses raw, unlabeled point clouds as inputs. The algorithm first applies modified Expectation-Maximization Gaussian Mixture Model (GMM) clustering to two input point clouds. It then performs change detection by comparing the GMMs using the Earth Mover’s Distance. The algorithm is validated quantitatively and qualitatively using a test dataset collected by an Astrobee robot in the NASA Ames Granite Lab comprising single frame depth images taken directly by Astrobee and full-scene reconstructed maps built with RGB-D and pose data from Astrobee. The runtimes of the approach are also analyzed in depth. The source code is publicly released to promote further development.

2023

The Impact of Time Step Frequency on the Realism of Robotic Manipulation Simulation for Objects of Different Scales

Minh Quang Ta*, Holly Dinkel*, Hameed Abdul-Rashid, Yangfei Dai, Jessica Myers, Tan Chen, Junyi Geng, and Timothy Bretl

In IEEE/RSJ IROS Workshop on Robotics and AI in Future Factory (RAFF), Oct 2023

Best Poster Finalist 🏆

Abstract PDF Poster Video

This work evaluates the impact of time step frequency and component scale on robotic manipulation simulation accuracy. Increasing the time step frequency for small-scale objects is shown to improve simulation accuracy. This simulation, demonstrating pre-assembly part picking for two object geometries, serves as a starting point for discussing how to improve Sim2Real transfer in robotic assembly processes.
TrackDLO: Tracking Deformable Linear Objects Under Occlusion with Motion Coherence

Jingyi Xiang, Holly Dinkel, Harry Zhao, Naixiang Gao, Brian Coltin, Trey Smith, and Timothy Bretl

IEEE Robotics and Automation Letters, Aug 2023

Abstract PDF Code Poster Video

The TrackDLO algorithm estimates the shape of a Deformable Linear Object (DLO) under occlusion from a sequence of RGB-D images. TrackDLO is vision-only and runs in real-time. It requires no external state information from physics modeling, simulation, visual markers, or contact as input. The algorithm improves on previous approaches by addressing three common scenarios which cause tracking failure: tip occlusion, mid-section occlusion, and self-occlusion. This is achieved through the application of Motion Coherence Theory to impute the spatial velocity of occluded nodes, the use of the topological geodesic distance to track self-occluding DLOs, and the introduction of a non-Gaussian kernel that only penalizes lower-order spatial displacement derivatives to reflect DLO physics. Improved real-time DLO tracking under mid-section occlusion, tip occlusion, and self-occlusion is demonstrated experimentally. The source code and demonstration data are publicly released.
Simultaneous Shape Tracking of Multiple Deformable Linear Objects with Global-Local Topology Preservation

Jingyi Xiang, and Holly Dinkel

In IEEE ICRA Workshop on Representing and Manipulating Deformable Objects, May 2023

Abstract PDF Code Poster Video

This work presents an algorithm for tracking the shape of multiple entangling Deformable Linear Objects (DLOs) from a sequence of RGB-D images. This algorithm runs in real-time and improves on previous single-DLO tracking approaches by enabling tracking of multiple objects. This is achieved using Global-Local Topology Preservation (GLTP). This work uses the geodesic distance in GLTP to define the distance between separate objects and the distance between different parts of the same object. Tracking multiple entangling DLOs is demonstrated experimentally. The source code is publicly released.
Multi-Agent 3D Map Reconstruction and Change Detection in Microgravity with Free-Flying Robots

Holly Dinkel*, Julia Di*, Jamie Santos, Keenan Albee, Paulo V.K. Borges, Marina Moreira, Oleg Alexandrov, Brian Coltin, and Trey Smith

In IAF International Astronautical Congress, Oct 2023

Abstract PDF Code Video

Assistive free-flyer robots autonomously caring for future crewed outposts—such as NASA’s Astrobee robots on the International Space Station (ISS)—must be able to detect day-to-day interior changes to track inventory, detect and diagnose faults, and monitor the outpost status. This work presents a framework for multi-agent cooperative mapping and change detection to enable robotic maintenance of space outposts. One agent is used to reconstruct a 3D model of the environment from sequences of images and corresponding depth information. Another agent is used to periodically scan the environment for inconsistencies against the 3D model. Change detection is validated \textita posteriori using real image and pose data collected by Astrobee robots in a ground testbed environment and from microgravity aboard the ISS. This work outlines the objectives, requirements, and algorithmic modules for the multi-agent reconstruction system, including recommendations for its use by assistive free-flyers aboard future microgravity outposts.

2022

Vela: A Data-Driven Proposal for Joint Collaboration in Space Exploration

Holly Dinkel*, and Jason Cornelius*

In IAF International Astronautical Congress, Oct 2022

Abstract PDF Supp

The UN Office of Outer Space Affairs identifies synergy of space development activities and international cooperation through data and infrastructure sharing in their Sustainable Development Goal 17 (SDG17). Current multilateral space exploration paradigms, however, are divided between the Artemis and the Roscosmos-CNSA programs to return to the moon and establish permanent human settlements. As space agencies work to expand human presence in space, economic resource consolidation in pursuit of technologically ambitious space expeditions is the most sensible path to accomplish SDG17. This paper compiles a budget dataset for the top five federally-funded space agencies: CNSA, ESA, JAXA, NASA, and Roscosmos. Using time-series econometric anslysis methods in STATA, this work analyzes each agency’s economic contributions toward space exploration. The dataset results are used to propose a multinational space mission, Vela, for the development of an orbiting space station around Mars in the late 2030s. Distribution of economic resources and technological capabilities by the respective space programs are proposed to ensure programmatic redundancy and increase the odds of success on the given timeline.
Wire Point Cloud Instance Segmentation from RGBD Imagery with Mask R-CNN

Holly Dinkel*, Jingyi Xiang*, Harry Zhao, Brian Coltin, Trey Smith, and Timothy Bretl

In IEEE ICRA Workshop on Representing and Manipulating Deformable Objects, May 2022

Abstract PDF Video

Perception of the shapes of deforming objects like wires enables their monitoring and manipulation by autonomous robots. This paper presents detection, classification, and instance segmentation of deformable wires from a cluttered scene in RGBD imagery. This work uses the Detectron2 implementation of Mask R-CNN trained with the PointRend mask head on the UIUCWires dataset as the framework for wire instance segmentation on RGB imagery, a method demonstrated to perform well for the instance segmentation task in previous work. In this work, the instance bitmask is directly used to segment individual object point clouds, an important step toward wire shape representation for manipulation tasks.
Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration

Tan Chen, Zhe Huang, James Motes, Junyi Geng, Quang Minh Ta, Holly Dinkel, Hameed Abdul-Rashid, Jessica Myers, Ye-Ji Mun, Wei-Che Lin, Yuan-Yung Huang, Sizhe Liu, Marco Morales, Nancy Amato, Katherine Driggs-Campbell, and Timothy Bretl

In IEEE ICRA Workshop on Collaborative Robots and Work of the Future, May 2022

Abstract PDF Video

Significant progress in robotics reveals new opportunities to advance manufacturing. Next-generation industrial automation will require both integration of distinct robotic technologies and their application to challenging industrial environments. This paper presents lessons from a collaborative assembly project between three academic research groups and an industry partner. The goal of the project is to develop a flexible, safe, and productive manufacturing cell for sub-centimeter precision assembly. Solving this problem in a high-mix, low-volume production line motivates multiple research thrusts in robotics. This work identifies new directions in collaborative robotics for industrial applications and offers insight toward strengthening collaborations between institutions in academia and industry on the development of new technologies.

2021

Development of a Private Space Sector in the U.S. and Russia

Jason Cornelius, Holly Dinkel, and Arzu Kurgan

In IAF Global Space Exploration Conference (GLEX), Jun 2021

Best Technical Presentation Finalist 🏆

Abstract PDF Supp Video

This work analyzes progress and challenges in the development of commercial space economies in the United States and Russia. Space development progress is characterized through examination of three production indicators for the space economies in each country and econometric analysis of the financial data for three new American private space companies. We also provide case studies of major challenges faced by three new private space companies and conjecture about the future of the private space industry through analysis of space education projects in each country. As leaders of space technological development and exploration, the U.S. and Russia are major stakeholders in the current and future global space economy. We find access to capital is a primary barrier to entrepreneurship. The results will help future founders more effectively navigate the complex financial, regulatory, and technological landscape of the space industry.

2016

Gas Filled RF Resonator Hadron Beam Monitor for Intense Neutrino Beam Experiments

Katsuya Yonehara, Robert Abrams, Holly Dinkel, Ben Freemire, Rolland Johnson, Grigory Kazakevich, Alvin Tollestrup, and Robert Zwaska

In International Particle Accelerator Conference, Jun 2016

Abstract PDF

MW-class beam facilities are being considered all over the world to produce an intense neutrino beam for fundamental particle physics experiments. A radiation-robust beam monitor system is required to diagnose the primary and secondary beam qualities in high-radiation environments. We have proposed a novel gas-filled RF-resonator hadron beam monitor in which charged particles passing through the resonator produce ionized plasma that changes the permittivity of the gas. The sensitivity of the monitor has been evaluated in numerical simulation. A signal manipulation algorithm has been designed. A prototype system will be constructed and tested by using a proton beam at the MuCool Test Area at Fermilab.

* Equal Contribution