PhD Proposal Presentation (pdf) - Georgia Institute of Technology

Outline 

Controlling a Passive Haptic Master During 

Teleoperation 

Ben Black 

George Woodruf School Mechanical Engineering 

Georgia Institute of Technology 

- 

Committee: 

Dr Book (chair) 

Dr Falcon (National Instruments) 

Dr Ferri (ME) 

Dr Jacko (ISyE) 

Dr Lee (ME) 

Dr Ting (BME) 

May 2, 2006 

Benjamin A. Black PhD Proposal - 5.2.06 1 / 59

Outline 

Part I - Background & Problem Statement 

Part II - Preliminary Work 

Part III - Proposed Work 

Full Outline 

1 Introduction and 

Background 

System Introduction 

2 Proposed Research 

3 Literature Review 

Teleoperation 

Haptics 

Passive Haptics 

Other Relevant Work 

Link 1 

Workspace Boundary 

Joint C 

Link 2 

Joint A & B 

Link 4 

Joint E 

Link 3 

Joint D 

Handle & Force Sensor 


Outline 




Full Outline 

4 Analysis of Force Generation 

Preliminary Discussion 

On-Off Actuation 

Multi-Brake Actuation 

Discussion 

5 Preliminary Analysis of Force 

Calculation Algorithm 

6 Preliminary Experiments 

Master Hardware 

Slave Hardware 

Control Software 

Testing 


Outline 




Full Outline 

7 Proposed Extensions of 

Preliminary Work 

Advanced Force Calculation 

Extension of Hardware and 

Software 

Human Testing 

8 Contributions 

9 Final Thoughts 

Acknowledgments 

Opportunity for Questions 


Introduction 

Proposal 

Lit. Review 


1 Introduction and Background 




Teleoperation 

Haptics 




Introduction 


Lit. Review 


Project Background 

Haptics - sense of touch 

Human-Computer Interface 

Augment user-interface by providing force feedback 

Teleoperation 

Remote control - master / slave 

Allows an expert user to remotely operate in dangerous 

environments 

Haptic Teleoperation - remote control with force feedback 


Introduction 


Lit. Review 


Haptic Categorizations 

Active Haptics 

Most prevalent group of haptic devices 

Actuated via motors, linear actuators, etc 

Allow energy to be added to the system 

Possibly dangerous to user 


Introduction 


Lit. Review 


Haptic Categorizations 


Energetically passive 

Energetically neutral - Cobots (Colgate) 

Dissipative - PTER & MR PTER (Book) 

Generated force can only oppose or redirect motion 


Introduction 


Lit. Review 


Motivation 

Why use passive haptic devices? 

Inherently safe! 

Can be low power 

Well-suited to guiding applications 

Less expensive in some cases 


Introduction 


Lit. Review 





Teleoperation 

Haptics 




Introduction 


Lit. Review 

Basic Proposal 

Problem Goals: 

Explore use of passive haptic master during teleoperation 

Understand the human response to a passive haptic device 

Produce control algorithm for providing haptic feedback 

Test algorithm through human-user teleoperation tasks 

All research will extend the small field of passive haptics 


Interesting Application 

Introduction 


Lit. Review 

Medical Haptics: 

Orthopedic assistance: Rossi & Boschetti 

Why passive haptic? 

Provides assistance 

Surgeon in complete control 


Introduction 


Lit. Review 

Teleoperation 

Haptics 







Teleoperation 

Haptics 




Introduction 


Lit. Review 

Teleoperation 

Haptics 



Teleoperation Research 

History and Applications: 

Father: Goertz (1954) 

Vertut (1976) 

Bejczy & Salisbury (1980) 

Current Interesting Applications: 

Space-teleoperation – Nohmi (2003) 

Remote surgery – Hayward (2004) 

Hazardous material handling – Lee & Park (2004) 

Etc, etc... 


Introduction 


Lit. Review 

Teleoperation 

Haptics 



Haptic Interfaces 

Beginning of popularity: 

Sherrick (1985) 

Klatzky (1987) 

Bajcsy & Campos (1991) 

Craig & Rollman (1999) 

Passivity: 

Hanneford & Ruy (2001) 

Lee & Li (1998) 

Passivity with respect to delay - wave variables 

Anderson & Spong (1989) 

Slotine & Niemeyer (1991) 

Extended: Ching (2005) 



Introduction 


Lit. Review 

Teleoperation 

Haptics 



Outside Georgia Tech 

Cobots - Colgate et. al (1996) 

Sakaguchi (2001) 

Matsuoka (2001) 

Wannasuphoprasit (2002) 

Cho - Force Manipulability Ellipsoid (2003) 



Introduction 


Lit. Review 

Teleoperation 

Haptics 



At Georgia Tech 

PTER (Passive Trajectory Enhancing Robot) 

Design - Charles (1997) 

Impedence Control - Gomes (1997) 

Velocity Field / Path Following - Swanson (2003) 

Mr PTER (Design and Control) - Reed (2003) 

Steerability - Gao (2005) 


Introduction 


Lit. Review 

Human-Computer Interaction 

Teleoperation 

Haptics 



Human Factors Research 

Fitts (1950s) 

Fitts’ Law - design of user interfaces 

behavioral response to stimuli 

Velocity Field / Path Following - Swanson (2003) 

Human Cognative Modeling 

Lochard and Murdock - signal detection theory (1970) 

Card, Morgan & Newell - Human Model Processor (1983) 

Anderson & Byrne - ACT Architecture (1993 / 2001) 

Kieras & Meyer - EPIC Architecture (1997) 


Introduction 


Lit. Review 

Teleoperation 

Haptics 



Further Research of Interest 

Wire-driven haptics: 

Melchiorri (1997) 

Haptic transparency: 

Sirithanapipat (2002) 

... 


Preliminary Analysys 

Analysis - Force Generation 

Analysis - Force Calculation 


The analysis of use of a passive haptic device can be divided into 

two sections: 

Force generation ability 

the actuation scheme to produce a specified force 

Force calculation 

determination of desired force magnitude and direction 








Discussion 





Discussion 

5 Preliminary Analysis of Force Calculation Algorithm 





Testing 








Discussion 

Single Degree of Freedom Paths 

Limitations of passivity 

Locking a single brake 

constrains endpoint to single 

DOF path 

1 single DOF paths for each 

actuator 

Non-redundent actuators 

yield unique paths 

p E 

p B 

p A 








Discussion 

Producible Forces 

Each brake force is 

perpindicular to its single 

DOF path 

Direction of force defined by 

joint velocity 

One unique force direction 

for each unique actuator 

Generated force must be π 2 

radians away from ⃗v 

−f E 

−f −f A 

B 

v 

f A f B 

f E 









Discussion 

v 

f h1 

f A 

f h2 

f B 

f E 

Simplest actuation scheme: 

Only actuates a brake if 

|f h − v| > π 2 

Actuate brake to produce 

a force closest to f h 

Difference between 

actuated force direction 

and f h is θ error 

NOTE: f h has not yet 

been defined 








Discussion 


f h1 

Next step in force production: 

v 

f E 

f A 

f h2 

f B 

Two zones (between two 

brakes & outside of 

brakes) 

Given e h1 actuate brake A 

Given e h2 actuate brakes 

A & B at the same time 


Using Two Brakes 

f2 

θ2 

θh 

fh 

θ1 

f1 







Discussion 

Use Jacobian to find force vector of unit actuation: 

[ ] T [ ] T [ ] T fx τ1 

= [J 

f y τ 12 ] −1 τ3 

+ [J 

2 τ 34 ] −1 

4 

where 1,2,3 & 4 represent brakes A,B,C or E 

sequentially set one brake torque to 1 and the 

others to 0 

yields unit actuation force for each brake 

Match the direction of resultant force: 

direction(f h ) = direction(af 1 + bf 2 ) 

f 1 & f 2 correspond to unit actuation of A,B,C or E 

tan(θ h ) = tan(af 1 θ 1 + bf 2 θ 2 ) 

Solve for actuation V 1 & V 2 given a magnitude, |f h |: 

V 1 = a|f h| 

√ 

a 2 +b 2 

Benjamin A. Black b|f | PhD Proposal - 5.2.06 26 / 59

Discussion 







Discussion 

Empirical values for judging haptic device: 

Direction: 

Judged by Average θ error 

Magnitude judged by: 

Producible endpoint forces 

NOTE: also a function of user input 








Discussion 

Average Angle Error (single brake actuation) 

e 3 

e 2 

e 1 

v 

producible forces 

Average over 2 unknown variables, θ v & θ h : 

Sum over regions: 

Avg (θ error ) = ∑ P (region) ∗ (AverageValue) 

or: 

Avg (θ error ) = ∑ regions 

∫ 

1 θupper 

∫ π 

πRegionSize θ lower 0 θ error dθ h dθ v 








Discussion 


e 2 

e 1 

π 

4 

θerror 

v 

e 3 

θ 3 θ 1 θ 2 π 

θ f 

For MRPTER: 

Avg (θ error ) = 2(θ2 3 +(π−θ 2) 2 )+((θ 1 −θ 3 ) 2 +(θ 2 −theta 1 ) 2 ) 

4π 








Discussion 


e 2 

e 1 

e 2 

e 1 

π 

4 

π 

4 

v 

θerror 

v 

v 

e 2 

e 3 

θerror 

e 3 

θ 3 θ 1 e 3 

θ 2 π 

θ f 

θ 3 








Discussion 

Average Angle Error (multi brake actuation) 

e 2 

e 1 

π 

4 

v 

e 3 

θ 3 

θ 1 θ2 

θ f 

π 








Discussion 

Average θ error for single brake 

actuation 

Average θ error for multi-brake 

actuation 


Force Output 







Discussion 

Maximum Force Output Defined by Using Maximum Brake Torque 

F Amax or F Bmax 

F Emax 









Discussion 






Testing 





Force Determination - teleoperation 

Determination of force to be displayed to user, f h 

Classic teleoperation: 

Virtual coupling 

Force based on difference in master & slave position 

Drawbacks for passive systems: 

Blind to human user input 

Overshoot error without damping term 

|ˆp s − ˆp m | 

ˆp m 

K 

Virtual Coupling 

ˆp s 





Force Determination - passive haptics 

Determination of force to be displayed to user, f h 

Typical for passive haptic devices: 

Steerability 

Velocity Field control 

Drawbacks for teleoperation 

Focused on controlling velocity direction 

Require knowledge of slave environment 








Testing 





Discussion 






Testing 





System-Level Overview 




Testing 

human 

operator 

force input 

force feedback 

haptic 

master 

sensor data 

brake actuation 

NI PXI 

controller 

#1 

master position 

UDP communication 

slave position 

NI PXI 

controller 

#2 

motor actuation 

slave position 

slave 

device 

4-Part system 

Human & Master 

Master control system 

Slave control system 

Slave device 








Testing 

MR PTER - Haptic Master 

Developed by Reed & Book 

Based on PTER by Charles, 

Swanson & Book 

Reconfigurable 4 or 5-Link 

manipulator 

Actuated by 3 or 4 

magneto-rheological brakes 

Link 1 

Joint C 

θ B 

θ A 

Joint A & B 

Link 4 

Joint E 

Controlled by National 

Instruments PXI hardware 

& LabVIEW software 

NOTE: not originally 

running in Real-Time 

Link 2 

Workspace Boundary 

Link 3 

Joint D – handle 








Testing 

Slave Device 

“Current” slave device: 

Linear motor – 1-DOF device 

Controlled using LabVIEW & PXI hardware 

Creates situation similar to previous path following research 








Testing 

Slave Device 

“Future” slave device: 

HURBIRT 

2-DOF system 

Controlled through LabVIEW & PXI system 

Kinematically similar to MR PTER 

Allows for more realistic teleoperation scenario 





Pure Teleoperation Video 




Testing 

play stop launch 








Testing 

Software for Experiment 

LabVIEW & 

LabVIEW Real Time 

Textural & graphical 

programming 

Concurrent loops 

running at different 

speeds and priorities 

Internet-based 

communication 








Testing 

First Experiment 

Simulated human input 

Constant force provided 

by weight attached to 

handle via string 

Experiment designed to 

test a large portion of 

workspace 


Video 







Testing 

play stop launch 








Testing 

Experimental Results – first experiment 

Control Position 

Difference 

Angle 

Error 

Total Time Unfinished 

Trials 

None 17.91% N/A 1.46 sec 0/18 

On-Off 2.26% 0.6 rad 16.53 sec 9/18 

From these results: 

ENACTIVE conference paper (November 2005) 

SYROCO conference paper (September 2006) 








Testing 

Introduction of CRS Testing Mechanism 

Problems with past testbeds: 

Non-Repeatability of informal test 

Non-Reality of first test 

Current Solution 

Use CRS 6-R robot to simulate human input 

Apply sinusoidal path of CRS robot applied changing p m 

Implement flexible coupling designed by Carwyn Jones 


Benjamin A. Black 

Step 1 - rigid connection 

CRS overpowered MR 

PTER 

control made no 

impact 

Step 2 - flexible rod 

single flexible rod 

produced reasonable 

results 

Step 3 - dynamic “mit” 

3-member spring / 

PhD Proposal damper - 5.2.06 system 

48 / 59 







Testing 

CRS Interface - thanks to Carwyn Jones / Matt Litman

Extensions 

Contributions 

Wrap-up 


Extension of Hardware and Software 

Human Testing 

7 Proposed Extensions of Preliminary Work 



Human Testing 






Extensions 

Contributions 

Wrap-up 



Human Testing 

Extensions 

Work over the next 12 months: 

Early human testing 

Advanced force calculation 

Extensions of hardware and software 

Analysis of force generation 

Human testing 


Extensions 

Contributions 

Wrap-up 



Human Testing 

Intermediate Testing 

Explore human resolution of force 

Compare virtual spring to actual spring 

Look at force generation of passive haptic device 

Attempt to understand resolution of force direction and 

magnitude 

Results to be published as ASME IMECE conference paper 

Point 3 

Point 1 Point 2 Point 4 

Point 5 

K 


Extensions 

Contributions 

Wrap-up 



Human Testing 

Force Calculation 

Divide haptic force into two parts: 

f h = f dynamic + f coupling 

Take dynamic forces into account when displaying force 

Improve transparency 

Expand coupling force, possible thoughts 

Include damping term 

Use slave velocity as path predictor 


Extensions 

Contributions 

Wrap-up 



Human Testing 

Upgrades to Hardware and Software 

Upgrade the hardware and software of the testbed, including: 

Extend to 2-DOF slave device 

Migrate control to a Real-Time operating system 

Upgrade and optimize control software for speed - 500Hz 

Verify National Instruments communication algorithms 


Extensions 

Contributions 

Wrap-up 



Human Testing 

Human Testing (for fine tuning controller) 

Possible details (VERY open for discussion): 

Realistic teleoperation task 

Multiple phases 

Early: preliminary test to influence control design 

Late: finalize discussion about control algorithms 

Late testing in two steps: 

Step 1: Compare 3-4 control algorithms 

Step 2: Refine gains and discuss aplication to specific tasks 


Extensions 

Contributions 

Wrap-up 



Human Testing 

Proposed Teleoperation Experiment 

Experiment designed to simulate tool task: 

Use MR PTER to control HURBIRT 

Place ”obstacle” in HURBIRT workspace 

Move tool from point A to point B 

Judge based on: 

speed of completion 

penetration into obstacle 

user workload (NASA-TLX) 


Extensions 

Contributions 

Wrap-up 

7 Proposed Extensions of Preliminary Work 



Human Testing 






Extensions 

Contributions 

Wrap-up 

Contributions 

Specific tools for comparing passive haptic devices 

Determination of passive force resolution by a user 

Identification of meaningful performance factors 

Quantification of performance factors 

Comparisons between different dissipative devices 

An algorithm for haptic feedback with a passive device 

Definition of useful passive haptic feedback 

Possible extension of Pahl & Beitz method to haptic control 

Verified control algorithm 

Combination of above to produce working feedback algorithm 

Verification of the haptic device’s force production 

Verification of the algorithm through human-user experiments 


Extensions 

Contributions 

Wrap-up 



I would like to thank... 

I appreciate the flexibility and input of my committee members and 

my advisor, Dr Book. Special thanks go to JD Huggins for his 

assistance with hardware and to Carwyn Jones for his extensions to 

my research. 

The work proposed here operates primarialy under a grant from 

the Academic Relations division of National Instruments whose 

support includes hardware, software and funding. In that, special 

thanks go to Dr. Jeannie Falcon, Morten Jensen, and Andy Deck 

for their support in the early stages of the project as well as their 

continued support throughout the process. 

Finally, I would like to thank my wife Amanda. 


Extensions 

Contributions 

Wrap-up 



Any Questions?

PhD Proposal Presentation (pdf) - Georgia Institute of Technology

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?