Identification of human control during walking

Indirect Identification of Human Control During Walking

Jason K. Moore j.k.moore19@csuohio.edu

Human Motion and Control Laboratory [hmc.csuohio.edu]

Cleveland State University, Cleveland, Ohio, USA

July 15, 2014

About

Currently a post doc in Ton van den Bogert's Human Motion and Control Lab at Cleveland State University
Graduate work was at UC Davis (Hubbard/Hess: Sports Biomech Lab) and TU Delft (Schwab: Bicycle Dynamics Lab) where I worked on the human control identification during bicycle balancing
Active in the Scientific Python Community (SymPy/PyDy)

Our big picture questions

What are the fundamental control mechanisms used during human gait?

Zero moment point control: Asimo, etc?
Optimal control: maximize stability, minimize energy, etc?
Something else?

What control mechanisms can powered prosthetics utilize to recreate able bodied human gait?

Maybe the previous are good choices, maybe not.
Is it possible to develop a data driven controller for a particular prosthetic that behaves like a human, defects and all?
What can common gait lab data from able-bodied humans tell us about the control mechanisms during gait? Can we measure what we really want to?
What kind of experiments can generate rich data needed to identify controllers?

How can we identify these controllers from large sets of gait data?

Maybe standard system identification methods?

Direct Approach
Indirect Approach

Our goal

Desired Improvements

Natural gait patterns
Balance

Additional Benefits

Quantification of control can possibly be used to assess patients with a new set of numbers

Clips collected from [1], [2], [3], and [4].

Our current approach

Collect common gait data from many gait cycles (500-1000)
During cycles, apply pseudo-random external perturbations to the human
Assume a simple time varying linear MIMO black box control structure
Find the best fit of the control model to the data with a direct approach
Test the resulting controller(s) in simulation and in actual devices (in progress)

Idealized Gait Feedback Control

Estimated

\(\varphi\): Phase of gait cycle
\(\mathbf{s}(t)\): Joint angles and rates
\(\mathbf{m}(t)\): Joint torques
\(w(t)\): Random belt speed

Unknown

\(\mathbf{K}(\varphi)\): Matrix of feedback gains
\(\mathbf{s}_0(\varphi)\): Open loop joint angles and rates
\(\mathbf{m}_0(\varphi)\): Open loop joint torques

Controller Equations

\[ \mathbf{m}(t) = \mathbf{m}_0(\varphi) + \mathbf{K}(\varphi) [\mathbf{s}_0(\varphi) - \mathbf{s}(t)] \\ \] \[ \mathbf{m}(t) = \mathbf{m}^*(\varphi) - \mathbf{K}(\varphi) \mathbf{s}(t) \] where \[ \mathbf{m}^*(t) = \mathbf{m}_0(\varphi) + \mathbf{K}(\varphi) \mathbf{s}_0(\varphi) \]

Gain Matrix: Planar Walker

Sensors

Assume that a lower limb exoskeleton can sense relative orientation and rate of the right and left planar ankle, knee, and hip angles.

\(\mathbf{s}(t) = \begin{bmatrix} s_1 & \dot{s}_1 & \ldots & s_q & \dot{s}_q \end{bmatrix} \) where \(q=6\)

Controls (plant inputs)

Assume that the exoskeleton can generate planar ankle, knee, and hip joint torques.

\(\mathbf{m}(t) = \begin{bmatrix}m_1 & \ldots & m_q \end{bmatrix} \) where \(q=6\)

Gain Matrix [Proportional-Derivative Control]

\( \mathbf{K}(\varphi) = \begin{bmatrix} k(\varphi)_{s_1} & k(\varphi)_{\dot{s_1}} & 0 & 0 & 0 & \ldots & 0\\ 0 & 0 & k(\varphi)_{s_2} & k(\varphi)_{\dot{s_2}} & 0 & \ldots & \vdots\\ 0 & 0 & 0 & 0 & \ddots & 0 & 0\\ 0 & 0 & 0 & \ldots & 0 & k(\varphi)_{s_q} & k(\varphi)_{\dot{s}_q} \\ \end{bmatrix} \)

Linear Least Squares

With \(n\) time samples in each gait cycle and \(m\) cycles there are \(mnq\) equations and which can be used to solve for the \(nq(p+1)\) unknowns: \(\mathbf{m}^*(\varphi)\) and \(\mathbf{K}(\varphi)\). This is a classic overdetermined system of linear equations that can be solved with linear least squares.

\[\mathbf{A}\mathbf{x}=\mathbf{b}\] \[\hat{\mathbf{x}}=(\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\mathbf{b}\]

\(n=20,m\sim=400,q=6,p=12\)
\(\mathbf{A}\) (48000 x 1560): joint angles and rates
\(\mathbf{b}\) (48000 x 1): joint torques
\(\mathbf{x}\) (1560 x 1): \(\mathbf{K}(\varphi)\) and \(\mathbf{m}^*(\varphi)\)

Random Belt Speed Variations

Gains: v=0.8 m/s

Gain variation with speed

How good is the model?

Can we put this controller into an exoskeleton?

Maybe

Issues

Difficult to validate the control law
Does the result reflect the inverse of plant or the controller?
In general, a plant is required to validate controller

Other approaches need to be explored

Indirect Identification

Choose a plant model
Choose a controller structure (black or grey box)
Formulate the close loop system equations of motion
Formulate a cost function based on minimizing the error in the simulated outputs and the gait measurements (marker coordinates, GRFs)
Search for optimal controller that reproduces the data
Potential optimization methods:
- simulation + genetic algorithms (shooting)
- discretization + nonlinear programming (direct collocation)

Controller Choice

Grey box

Feedback/feedforward elements
Muscle-reflex control [Geyer et. al 2010]
Neuro time delay

Black box

Full state/output feedback
Gait phase gain scheduling
Neural networks [Tan et. al 2014]

Shooting

Free parameters

\( \mathbf{\theta} = \mathbf{x}_0, p_1, ..., p_q\), \(q = 100\) to \(300\)

Gradient free parallel optimization methods

Covariance Matrix Adaptation [Hansen 2006]
Biogeography-based [Simon 2008]

Cost function

Minimize model error wrt measurements \( J(\theta) = \sum_{i=1}^M || y_{mi} - y_i || \)
Penalize large trajectory excursions
Penalize rapid energy use

Computational Costs

1000-3000 iterations for 10 second simulation
10-12 hr computational time for convergence
See Wang et.al 2010 and Geijtenbeek et. al 2013

I need a 25x to 50x simulation speedup (480 second simulation) to have 1 day turn around on solutions

Direct Collocation Variables

\(\mathbf{x}\): state vector (joint angles and angular rates)
\(\mathbf{u}^u\): unknown input vector (open loop joint torques)
\(\mathbf{u}^k\): known input vector (belt speed perturbation)
\(p_1, \ldots, p_q\): controller parameters
\(\mathbf{y}_m\): measured output variables (marker locations, ground reaction loads)
\(\mathbf{y}\): model output variables (marker locations, ground reaction loads)

\(M\): # measured time samples
\(N\): # discretized samples (50-100 for gait cycle)
\(n\): # states (~18 for "simple" planar walker)
\(m\): # unknown inputs (~9 open loop joint torques)
\(p\): # known inputs (1 belt speed)
\(q\): # free model constants (100-300 controller parameters)

Direct Collocation Model Formulation

Free Parameters

\( \theta = \mathbf{x}_1, \ldots, \mathbf{x}_N, \mathbf{u}^u_1, \ldots, \mathbf{u}^u_N, p_1, \ldots, p_q \) ~1M

Cost Function and Gradient

\( J(\theta) = \sum_{i=1}^M || y_{mi} - y_i || \)

Constraints and Jacobian

\(0 = \mathbf{f}(\dot{\mathbf{x}}, \mathbf{x}, \mathbf{u}, t)\) has to hold at each discrete time instance

Backward Euler Discretization

\( 0 = \mathbf{f}_j(\mathbf{x}_j, \mathbf{x}_{j-1}, \mathbf{u}^k_j, \mathbf{u}^u_j, \mathbf{u}^u_{j-1}, p_1, \ldots, p_q) \in \mathbb{R}^n \quad j=2,\ldots,N \)

\(N \approx 1,000,000\)

Direct Collocation Solution

Plant Model

Simple 2D torque actuated 10 DoF planar walker (Opensim gait10dof18musc)

Control Model

A simpler version of the phase muscle-reflex controller in Wang et. al 2010
Simple PD gain gait phase scheduled controller + optimal open loop joint torques

Data

2D marker locations
Ground reaction loads

Solvers

Interior Point Optimization: IPOPT (included in Simbody)
Sequential Quadratic Programming: SNOPT
Other ...

Direct Collocation Hurdles

Computational speed with ~1,000,000 constraints
Efficient sparse Jacobian computations: Simbody/Opensim only offer numerical gradients. Automatic differentiation is an option.
Local minima
Initial guesses

Plan

Week 1-2: Develop (or use) a closed loop planar walker system in Simbody/Opensim
Week 3: Evaluate which optimization method is most tractable
Week 3-4: Implement the identification method
Week 5: Hack solutions to everything that doesn't work as planned