AnyMoCap framework based on C3D data with markers' dropouts and without residuals

[depierie (Enrico De Pieri)]

Hi there,

I am working on a set of motion capture data which present some occasional gaps in the markers' coordinates, and I don't have any information on markers residuals.
I have the markers trajectories stored as csv files, and I am building my own c3d files in order to use the standard AnyMoCap workflow. For each marker in the c3d file, I'm setting the residual to 0 when the measurement is valid, and to -1 when the marker drops out.

I would like to understand more in details how the specific settings in AMS can deal with markers drop-outs and residuals.

• In MOCAP_C3DSETTINGS "C3DSettings.any", I can set GapFillUsingResidualsOnOff = On/Off;

• In the MOCAP_MARKER_PROTOCOL_FILE I can create specific drivers for each marker with the option UseC3DWeightResiduals = On/Off.

What's the intended use of these two parameters?

For GapFillUsingResidualsOnOff I couldn't find any documentation.
I tested it out for a few different c3d files, and if it's turned OFF, the marker tracking fails whenever there's any gap (short or long) at any point in any of the markers. The simulation will run successfully only if there are no gaps in the markers at all.
When it's turned ON, the simulations run even if there are certain gaps in the marker data, as long as they are not in the first frame of the trial.
From what I understood it fills the gaps already when reading the input (e.g. Main.ModelSetup.C3DFileData.Points.Markers.*.Pos)
But what kind of filling does it do exactly?

For UseC3DWeightResiduals I found some information in the AMMR changelog:
All MoCap model examples now use the option (UseC3DWeightResiduals=ON) in the marker protocols. This ensures that the marker weights are reduced to zero when a marker drops out
(i.e. when the residual value in the C3D file becomes negative).

If I leave UseC3DWeightResiduals=ON in my application, this causes some sudden accelerations of the foot segment when the heel marker drops out, leading to sudden peak in the predicted forces throughout the lower limb.
When I turn UseC3DWeightResiduals=OFF, the sudden accelerations of the foot and associate force errors do not occur.
My theory at the moment is that when the weight of the heel marker driver is suddenly set to zero, the tracking of the foot is driven only by the remaining markers on the hindfoot, which are generally less precise and were therefore manually assigned a lower weight until that time point. The lower weights on these markers meant that there was a larger tolerance between the virtual and the actual positions of these markers up until that point. In other words, when the heel marker drops out, the position of these markers is not perfectly consistent with the kinematic tracking that was done up to that point, and this causes a jump when the foot is suddenly being driven by this new set of markers.

In general, I would like to ask you what is the best approach for dealing with markers data.
I assume that filling short gaps prior to run the analysis in AMS is always advisable.
For any longer gaps, the issue is not so trivial.
On one hand we could just fill the gap anyway, and trust the fact that this artificial data points are realistic (very motion- and marker-dependent, not universally applicable);
or we could switch the markers drivers temporarily off with the weighing function, hoping that there are no kinematic inconsistencies in the transitions (but they can occur, as reported above);
or we could just exclude the specific markers from the whole trial, although this seems not ideal, as we would "waste" meaningful and accurate information for rest of the trial.

Is there any way we can make the best out of all the marker data we have, without excluding certain markers entirely and without causing these inconsistencies?
Did anyone ever tried a different function instead of AnyFunSquareWave to set up the markers' weights? I'm wondering if something smoother before and after the marker dropout would solve some of the kinematic inconsistencies, while preserving all the meaningful information away from the gap.

And re the residuals' values actually needed for this? Or is the CreateMarkerDriver class only checking if they are negative? This would confirm that I'm building my c3d files correctly.

Thanks for the help!

Enrico

[melund (Morten Enemark Lund)]

Hi Enrico

These are all very good questions.

If you look at the documentation for GapFillUsingResidualsOnOff it says:

Normally the values of the markers are zero in the C3D file when markers drop out. That presents a problem when lowpass filtering the data because of the sudden changes in the data. Setting
GapFillUsingResiduals will apply a linear interpolation when markers are dropout (in the intervals where residual is -1). Something like my sketch below:

This a poor-man's gap filling but it solves a lot of problems with filtering the data. So you should probably leave that on.

But you should probably not trust the marker data in those dropout regions unless the dropouts are very short. If you recorded data with a system like Qualisys or Vicon you can gap-fill in their software using a manual procedure and with much better spline curves. I highly recommend that to anyone who reads this. But you can't really do that as you are creating your C3D files from a CSV data.

If gap-filling is not an option. For example with longer dropouts then you can use UseC3DWeightResiduals to apply a zero weight to the marker in the period when the residual is -1.

But that can also be a problem as you correctly deduced. When the marker errors is large and the marker suddenly drops out then other markers will take over causing sudden acceleration in your data. That will manifest itself as spikes in the muscle forces.

I'm wondering if something smoother before and after the marker dropout would solve some of the kinematic inconsistencies, while preserving all the meaningful information away from the gap.

Yes. That is exactly the solution. It was originally the AnyInputC3D class that applied the weight functions based on residuals. Now it is done by the CreateMarkerDriver class template, but the template still uses a configuration (AnyInputC3D.WeightTransitionTime) in the C3D object when applying the fade-in fade-out time of the maker weight.

The variable WeightTransitionTime currently defaults to 0.1 second. Try to increase it to 1 or 2 seconds. This will of course mean that your markers are affected for a much longer period. But it may prevent some of the high acceleration.

As a side note: We will probably change this in the AMMR examples, so fewer people run into the same issue as you Enrico. Maybe accompanied with a warning if the data has drop-outs. High quality simulations usually require that drop-outs are dealt with in the MoCap software.

And re the residuals' values actually needed for this? Or is the CreateMarkerDriver class only checking if they are negative? This would confirm that I'm building my c3d files correctly.

But that is actually what the CreateMarkerDriver class template is doing. It only uses the on/off values of the residuals to add the marker weights. The weights transitions between 0 and what ever the users specified. Maybe the name UseC3DWeightResiduals is misleading. Any good ideas for new name?

[depierie (Enrico De Pieri)]

Hi Morten,

Thank you very much for the clear explanation!

So the good news is that it is possible to fill the gaps in Vicon Nexus even if I created my own c3d files from scratch. I still have to check whether all the gap-filling options are available (as some might require to reconstruct the model kinematics with a template skeleton), but at least it is possible and I can visually assess the quality of the filling.
Gap filling in Nexus also seems to automatically set the residuals to 0 instead of -1, which is quite convenient.

Besides that, I will keep the GapFillUsingResiduals=On , just in case I miss, or I can't fill, the occasional gap.

I also tried to solve the sudden accelerations with Main.ModelSetup.C3DFileData.WeightTransitionTime=1; when UseC3DWeightResiduals=On and it worked well, so thanks for the tip!
I think it's a nice way to retain most of the kinematic information from the markers. If the alternative is to exclude some marker(s) altogether, the long transition times are still a better choice.

One last thing that might be important to clarify for other users as well, is what happens when both GapFillUsingResiduals=On and UseC3DWeightResiduals=On. AMS fills the gaps, but then still temporarily deactivates the marker drivers because the residuals are still set to -1.

Good question with the name. Maybe MarkerDriverTempSwitchOffFromC3DResiduals? A bit too long, but more self explanatory?

Thanks again for everything!
Enrico