Need help interpreting data given in *_rowingData.csv

[klamp169]

A little worried some of the questions below have been answered elsewhere, I post them anyhow:)

While trying to make a more detailed comparison between ORM and rwnp, I encountered a few things that puzzle me about the data in the ORM CSV datafile:

1. What is the meaning of the row in the table with Stroke Number 0?
2. The line in the csv file containing data for Stroke Number 1 has quite a few NaN values, for instance in the Power (watts) column Why is that? The PM2 monitor does give a value for the first stroke power.
3. Related to the previous question: I do find evidence that the very first stroke data is appearing in the row labeled with Stroke Number 2 ?! Am I mistaken?
4. At the end of the row, I see a similar unclarity: There seems to be one (or maybe even 2) stroke(s) missing.
5. At the end, there are also two lines with the same stroke number. The background for this is unclear to me.
6. What is the purpose of the "Index" column (B) and the very first column (identical to "index" column)?
7. Drive length seems to be unrealistically large. (we discussed this before in a different thread)
8. What is the definition of AverageDriveForce (N)? Or rather, how is it calculated?
9. The CSV file contains data in columns labeled Handle_Force_(N), Handle_Velocity_(m/s), Handle_Power_(W). The given values are sampled at a specific time. What are the corresponding time (or stroke distance) values?
10. What is the difference between columns "DistanceMeters" and " Horizontal (meters)"

Thanks a lot for your consideration.
I am fully aware that "a fool can ask more questions than a wise man can answer."

[JaapvanEkris]

1. What is the meaning of the row in the table with Stroke Number 0?

That effectively is the beginning of the first stroke. All subsequent reports are at the end of a recovery. If you don't record it, you'll effectively miss the time/distance of the first stroke. Most postprocessing requires an initial (0 meters and 0 time) starting point.

This becomes clearer if you look at FIT-files or TCX-files which are processed by Strava: a missing zerod startingpoint will throw off the totals.

2. The line in the csv file containing data for Stroke Number 1 has quite a few NaN values, for instance in the Power (watts) column Why is that? The PM2 monitor does give a value for the first stroke power.

Because Concept2 doesn't validate their data, and we do. The first stroke typically contains only half the drive as it needs to clear the minimum flywheel speed. However, metrics like pace, power, etc. are only defined for a full stroke, not a partial one. Above that, these metrics use a formula designed for steady state (i.e. a stable stroke, where angular velocity is close together at start/finish stroke) which isn't the case at the hefty acceleration at the start. So Concept2 messes up here.

3. Related to the previous question: I do find evidence that the very first stroke data is appearing in the row labeled with Stroke Number 2 ?! Am I mistaken?

Yes, you are mistaken :). But more seriously: what kind of evidence exactly?

4. At the end of the row, I see a similar unclarity: There seems to be one (or maybe even 2) stroke(s) missing.

Start and stop behaviour on a PM5 is weird, as it actually seems to depend on voltage, not frequency. So in validation tests we explicitly always use a time/distance limited session.

5. At the end, there are also two lines with the same stroke number. The background for this is unclear to me.

Typically, it is the session end report, which should differ in time and distance.

6. What is the purpose of the "Index" column (B) and the very first column (identical to "index" column)?

It is a technical column used by RowsAndAll to parse the data.

7. Drive length seems to be unrealistically large. (we discussed this before in a different thread)

Please realize that our definitionbof 'drive' differs from Concept2's. For Concept2, the drive is when the flywheel accelerates. For ORM it is when there is any force present aside the dragforce. Typically this adds about 10% of drivetime and some drivelength.

8. What is the definition of AverageDriveForce (N)? Or rather, how is it calculated?

It is the force on the handle, based on Torque and sprocket size, with a time-based average across the drive.

9. The CSV file contains data in columns labeled Handle_Force(N), Handle_Velocity_(m/s), Handle_Power_(W)_. The given values are sampled at a specific time. What are the corresponding time (or stroke distance) values?

The total length is the dtivelength, similar to a PM5. Datapoints are collected per magnet impulse.

10. What is the difference between columns "DistanceMeters" and " Horizontal (meters)"

None, but RowsAndAll likes both of them being present.

With 0.9.7 (release this weekend) we'll add the RowsAndAll/RowingData fields to the FIT-file, as we've agreed among several major parties in the rowing world.

[klamp169]

1. What is the meaning of the row in the table with Stroke Number 0?

That effectively is the beginning of the first stroke. All subsequent reports are at the end of a recovery. If you don't record it, you'll effectively miss the time/distance of the first stroke. Most postprocessing requires an initial (0 meters and 0 time) starting point.

Why report at end of recovery? Concept2 reports at end of drive, AFAIK.
So postprocessing is based on CSV file? Why not disconnect creation of FIT/TCX from structure of CSV file. It could remove ambiguity in the "Stroke Number" column. You could also get rid of the first two columns.

This becomes clearer if you look at FIT-files or TCX-files which are processed by Strava: a missing zerod startingpoint will throw off the totals.

2. The line in the csv file containing data for Stroke Number 1 has quite a few NaN values, for instance in the Power (watts) column Why is that? The PM2 monitor does give a value for the first stroke power.

Because Concept2 doesn't validate their data, and we do. The first stroke typically contains only half the drive as it needs to clear the minimum flywheel speed. However, metrics like pace, power, etc. are only defined for a full stroke, not a partial one. Above that, these metrics use a formula designed for steady state (i.e. a stable stroke, where angular velocity is close together at start/finish stroke) which isn't the case at the hefty acceleration at the start. So Concept2 messes up here.

Concept2 messes up... Yet, they set the standard. It does not matter too much though: only the reporting of meters rowed is essential, IMO.

I do not like power/pace reporting of Concept2 either, see last figure/discussion in this thread

3. Related to the previous question: I do find evidence that the very first stroke data is appearing in the row labeled with Stroke Number 2 ?! Am I mistaken?

Yes, you are mistaken :). But more seriously: what kind of evidence exactly?

I will come back to this later.

4. At the end of the row, I see a similar unclarity: There seems to be one (or maybe even 2) stroke(s) missing.

Start and stop behaviour on a PM5 is weird, as it actually seems to depend on voltage, not frequency. So in validation tests we explicitly always use a time/distance limited session.

...depend on voltage... Indeed remarkable! (I only use a PM2 and I have no experience with PM5)

5. At the end, there are also two lines with the same stroke number. The background for this is unclear to me.

Typically, it is the session end report, which should differ in time and distance.

See my comment under 1.

6. What is the purpose of the "Index" column (B) and the very first column (identical to "index" column)?

It is a technical column used by RowsAndAll to parse the data.

See my comment under 1.

7. Drive length seems to be unrealistically large. (we discussed this before in a different thread)

Please realize that our definitionbof 'drive' differs from Concept2's. For Concept2, the drive is when the flywheel accelerates. For ORM it is when there is any force present aside the dragforce. Typically this adds about 10% of drivetime and some drivelength.

I will come back to that later, based on simulated data.

8. What is the definition of AverageDriveForce (N)? Or rather, how is it calculated?

It is the force on the handle, based on Torque and sprocket size, with a time-based average across the drive.

I think a better definition would be: $F_{AverageDriveForce}=\frac{1}{l_{drive}}\int Fdx$, where the integral is taken over the drive length. This is the force that should be sustained during the entire drive to arrive at the same energy per stroke as the real force.

9. The CSV file contains data in columns labeled Handle_Force(N), Handle_Velocity_(m/s), Handle_Power_(W)_. The given values are sampled at a specific time. What are the corresponding time (or stroke distance) values?

The total length is the dtivelength, similar to a PM5. Datapoints are collected per magnet impulse.

Clear!

10. What is the difference between columns "DistanceMeters" and " Horizontal (meters)"

None, but RowsAndAll likes both of them being present.

See comment under 1

With 0.9.7 (release this weekend) we'll add the RowsAndAll/RowingData fields to the FIT-file, as we've agreed among several major parties in the rowing world.

I am looking forward to the update!

[JaapvanEkris]

1. What is the meaning of the row in the table with Stroke Number 0?

That effectively is the beginning of the first stroke. All subsequent reports are at the end of a recovery. If you don't record it, you'll effectively miss the time/distance of the first stroke. Most postprocessing requires an initial (0 meters and 0 time) starting point.

Why report at end of recovery? Concept2 reports at end of drive, AFAIK.

Not sure about that. You need a dragfactor to calculate anything related to linear physics. Can't do that without recovery.

In fact, prof. van Holst (the first to reverse engineer the PM) was not sure either. ORM internally calculates metrics at both moments, and reporting is done once a full drive and full recovery complete, making a full stroke. And looking at the structure of the PM5 BLE messages, I suspect they might do the same internally.

As linear velocity is calculated across a complete stroke, and isn't in any way defined for a partial stroke, there is no way around this.

So postprocessing is based on CSV file? Why not disconnect creation of FIT/TCX from structure of CSV file. It could remove ambiguity in the "Stroke Number" column. You could also get rid of the first two columns.

You realize that all file formats are used for post processing? RowingData goes to RowsAndAll, FIT-files go to intervals.icu. The same goes for the PM5 emulation and the ANT+ data: we standardise definitions across all consumers. Creating a consistent set of metrics is quite essential if people use multiple platforms.

There is absolutely no ambiguity in stroke numbers? You need to have a start indicator as some post-processors simply substract the first recorded time/distance from the last recorded time/distance to get the total session time/distance. So stroke 0 is a necessity, regardless of file format. Sites like Strava are notorious for that, as they also recalculate velocities with weird consequences.

When crossing a threshold distance, be it a split/interval/session boundary, there is no reason to increase the stroke number as the stroke hasn't completed yet (which will happen milliseconds later)

Apps like Anthropic Claude have no issue in interpreting RowingData csv's either, so we aren't that exotic.

I will come back to that later, based on simulated data.

8. What is the definition of AverageDriveForce (N)? Or rather, how is it calculated?

It is the force on the handle, based on Torque and sprocket size, with a time-based average across the drive.

I think a better definition would be: F A v e r a g e D r i v e F o r c e = 1 l d r i v e ∫ F d x , where the integral is taken over the drive length. This is the force that should be sustained during the entire drive to arrive at the same energy per stroke as the real force.

Once the work per stroke metric has proven itself in production (0.9.7 feature) it might be an option, not sooner.

[JaapvanEkris]

Thanks a lot for your consideration. I am fully aware that "a fool can ask more questions than a wise man can answer."

My key approach with OpenRowingMonitor is that as soon we stop questioning the status quo, we stop innovating. Our jumps in development were triggered by seeing data that we didn't like, realising we had to fix something, and discovering we needed a quantum leap to do it.

A few key examples:

• We used to use a numerical approach to determine angular velocity and angular acceleration. However, the angular acceleration was unstable in the presence of noise, making the force curve erratic. That forced us to move to the current regression-based approach, which accepts noise as a fact of life, and keeping focus on the trend, reliably producing force curves;
• Pre 0.9.7 we knew structual noise was an issue. But we accepted it, as fixing it required a predetermined (impossible to calculate) filter that needed to be synchronised to the actual flywheel position (which is extremely hard). However, current dynamic approach is way more elegant and is just as effective.

We still are not perfect. The PM5 has its issues (see the research of Ulm university), especially at startup and at imperfect repeating strokes. But I also see in experimental builds that we don't take in account magnetic drag (i.e. the power generation by the 12 pole magnet). Might be extremely C2 specific, but pushing boundaries is what we should do. And without questions from intelligent people, that is impossible. So keep challenging us.

[klamp169]

2026-02-20_09-39-57_rowingData.xlsx
This ORM-based CSV datafile has been extended with data from rwnp (greenish columns) and my PM2 monitor (orange column, evaluated by videoing the PM2 and transcribing).

I shifted the data in column M and N up and down 1 or 2 strokes to find the best match with the "Power (watts)" column by inspecting the graph that can be seen by scrolling down in the spreadsheet. This led me to the suggestion that the first stroke seems to be located in the row indicated with "Stroke Number 2". (I hope that this procedure is sufficiently clear...)

At the end of this short row of 46 strokes, you can see that both rwnp and PM2 seem to have two additional strokes when comparing with ORM data.

This side by side comparison shows that all PM-systems lead to similar results, except for the first and last two strokes. Strong changes in power seem to lead to larger discrepancies between ORM and PM2/rwnp.

The other graph (scroll right in spreadsheet) shows a side by side comparison between rwnp and ORM on drag factor. The agreement is quite remarkable, as is the somewhat unsteady stroke-to-stroke value, that we have noticed before. But please notice that column with k_rwnp is shifted up one stroke with respect to all other greenish columns to get to a good match in the graph!

All this seems to indicate that we (I?) should better define what we mean with a stroke, especially at start, end and also when temporarily halting the row for 1 or 2 minutes. (The definition I use now is: Whenever I detect a "drive end"/"recovery start" I have a new stroke. For the first stroke I use a default value for drag factor. This seems to be consistent with the PM2 behaviour.)

The figure above shows a time-plot of the effective force on the flywheel. In this figure "drive end"/"recovery start" is indicated with tiny green circular markers. The red markers indicate "drive start"/"recovery end". At every green marker I evaluate a new stroke and at every red marker I evaluate a new value for the drag factor (default value for very first one), which is used in the evaluation of the subsequent stroke. So my first stroke comprises only a drive and no recovery!

For reference purposes, I also include the csv file I used as input to ORM (with 6 pulses per rev).

rwnp26-02-20_10-38-48_6p.csv.zip

[klamp169]

I am aware of the publications you mention. Most of the deviations Treff et. al. describe can simply be attributed to the fact that Concept2 only reports the power lost in air drag. The neglected dynamic term is especially important at start and unsteady rowing. It is remarkable that the authors do not seem to be aware of this as they do not mention it in their paper.
All this closely relates to my last contribution in: Simulation to generate Performance Monitor test data
Could you comment on the reported differences in the last graph in that thread?

You mention quite a few ORM-parameters that have to be set identical to PM5. I do not own a PM5, but AFAIK the PM5 does not make use of these specific parameters (except perhaps inertia and sprocket radius). Am I mistaken/misunderstanding?

You also mention the 0.03% deviation with PM5. As you mentioned earlier, this relates to the distance rowed. This quantity has the major advantage that it depends only on the drag factor and the number of magnet pulses detected. We both agree very closely on the value of the stroke-by-stroke drag factor (see the graph in the spreadsheet file I sent), and counting pulses is easy. And I am confident that Concept2 takes an identical approach to calculate the distance rowed.

[JaapvanEkris]

I am aware of the publications you mention. Most of the deviations Treff et. al. describe can simply be attributed to the fact that Concept2 only reports the power lost in air drag. The neglected dynamic term is especially important at start and unsteady rowing. It is remarkable that the authors do not seem to be aware of this as they do not mention it in their paper.

I talked to Gunnar (Treff) about it, their opinion was that it would be too much speculation of how a PM5 would work internally, and that it would distract from their more external findings,

All this closely relates to my last contribution in: Simulation to generate Performance Monitor test data Could you comment on the reported differences in the last graph in that thread?

I'm currently relaxing after the release of 0.9.7 and I have four book chapters to complete before June, so I get to it when time is there. Can't give a final time on that one I'm afraid.

You mention quite a few ORM-parameters that have to be set identical to PM5. I do not own a PM5, but AFAIK the PM5 does not make use of these specific parameters (except perhaps inertia and sprocket radius). Am I mistaken/misunderstanding?

Indirectly they have to be. Some parameters we know they exist, like maximumTimeBetweenImpulses, as there is a reported BLE-state "Below_Minimum_Flywheelspeed" and we see no distance/time change when a flywheel is pulled slowly. But other parameters must be set to in a certain way have ORM behave like a PM5. They coud be set to 0, but even then, that is a setting of a ORM-parameter that must set in a way to have ORM match a PM5's behaviour.

Please realize, we want our physics model to be correct, and matching it to an oracle-system (the PM5) is a good way of doing it. So for us, a 100% match is our ambition, but not a goal. Our goal is to have a reliable universal rowing monitor. Especially as Ulm's criticism is valid, and we might be able to actually outperform a PM5 on that aspect.

You also mention the 0.03% deviation with PM5. As you mentioned earlier, this relates to the distance rowed. This quantity has the major advantage that it depends only on the drag factor and the number of magnet pulses detected. We both agree very closely on the value of the stroke-by-stroke drag factor (see the graph in the spreadsheet file I sent), and counting pulses is easy. And I am confident that Concept2 takes an identical approach to calculate the distance rowed.

I don't known. On a pure air-drag based approach that might be the case. But technically a Model D is a hybrid rower, and C2 certainly knows what they are doing, so it might be part of the underlying physics. It might explain behaviour we are seeing in the dragfactor (the deviation of the dragfactor seems to vary based on flywheel speed, which might make sense if C2 was compensating for a static magnetic drag).

[klamp169]

I talked to Gunnar (Treff) about it, their opinion was that it would be too much speculation of how a PM5 would work internally, and that it would distract from their more external findings,

Understandable. But it would have helped in understanding their findings.

I'm currently relaxing after the release of 0.9.7 and I have four book chapters to complete before June, so I get to it when time is there. Can't give a final time on that one I'm afraid.

I will give it a spin. Have a great easter-weekend!

I don't known. On a pure air-drag based approach that might be the case. But technically a Model D is a hybrid rower, and C2 certainly knows what they are doing, so it might be part of the underlying physics. It might explain behaviour we are seeing in the dragfactor (the deviation of the dragfactor seems to vary based on flywheel speed, which might make sense if C2 was compensating for a static magnetic drag).

An interesting metric for the comparison with PM5 might be: RMSE of stroke power.

...hybrid rower...: Good point. My model C does not have this complication, since it does not have a generator. Curious to know how much power/torque this generator consumes.

[klamp169]

Hey Jaap,
Hope your chapters are progressing well!

Just to cath up on things:

I talked to Gunnar (Treff) about it, their opinion was that it would be too much speculation of how a PM5 would work internally, and that it would distract from their more external findings,

Understandable. But it would have helped in understanding their findings.

Rereading parts of the Treff et al. paper, I found it's there after all in the section "Underlying Mechanisms". The paper focuses on stroke to stroke power measurement. For using the rower in a competitive environment, the measurement of rowing distance is of more importance, I think. This quantity does not suffer from "uneven rowing problems".

I'm currently relaxing after the release of 0.9.7 and I have four book chapters to complete before June, so I get to it when time is there. Can't give a final time on that one I'm afraid.

I will give it a spin. Have a great easter-weekend!

I updated my ORM to 0.9.7 and found that it worked like a charm. The only operational difference I experienced was on the user interface.

I don't known. On a pure air-drag based approach that might be the case. But technically a Model D is a hybrid rower, and C2 certainly knows what they are doing, so it might be part of the underlying physics. It might explain behaviour we are seeing in the dragfactor (the deviation of the dragfactor seems to vary based on flywheel speed, which might make sense if C2 was compensating for a static magnetic drag).

An interesting metric for the comparison with PM5 might be: RMSE of stroke power.

...hybrid rower...: Good point. My model C does not have this complication, since it does not have a generator. Curious to know how much power/torque this generator consumes.

A coast-down of the ModelD might give a clue. For the model C coast-down leads to close to perfect linear behavior of $\dot\omega$ versus $\omega^2$, with an estimate of the drag factor from the slope. Extra "magnetic" drag from the generator might show up as a deviation from linearity.

[JaapvanEkris]

Hey Jaap, Hope your chapters are progressing well!

Not as fast as I hoped, bu that is writing for you.

Rereading parts of the Treff et al. paper, I found it's there after all in the section "Underlying Mechanisms". The paper focuses on stroke to stroke power measurement. For using the rower in a competitive environment, the measurement of rowing distance is of more importance, I think. This quantity does not suffer from "uneven rowing problems".

Yeah, that is why in validation we focus on this as well. In essence, once you get the drag stable, distance is essentially reduced to pulse counting. So from reducing complexity, it is the first angle of attack as we are seeing the same pulses. This is plus or minus the startup pulses, which we influence by the maximumTimeBetweenImpulses setting. So for distance there are only two factors at play: maximumTimeBetweenImpulses and FlywheelInertia. I use series of sessions (a 4 2.5K's, 4 5K's, 5 10K's, 4 21K's) at the exact same dragfactor setting. If the absolute error between PM5 and ORM gets bigger as the distance increases, the FlywheelInertia is too low. If it is a constant error, it is caused by maximumTimeBetweenImpulses.

A coast-down of the ModelD might give a clue. For the model C coast-down leads to close to perfect linear behavior of ω ˙ versus ω 2 , with an estimate of the drag factor from the slope. Extra "magnetic" drag from the generator might show up as a deviation from linearity.

I think I found a way of filtering this. As our drag calculation looks for the linear relation between currentDt over time over quite some datapoints (200+ for a RowErg), we could model the remainder here. I already got a theoretical solid solution, that would be easily collapsed once the recovery completes (preventing a high CPU load on the start of the drive) that is based on WLS. Practice will have to show wether it is a stable measurement (which it should be, as the physics wouldn't change over time).

By the way: I think you are correct in your thinking that a PM5 reports on the end of the drive. When I look at my screen, pace is only updated at the end of the drive, never at the end of the recovery. I don't think we will go that way with ORM, as it weirdly splits up strokes.

I can imagine you want the recordings to match. Then you'll need to change this line

openrowingmonitor/app/recorders/rowingDataRecorder.js

Line 83 in b380437

case (metrics.metricsContext.isDriveStart):

to case (metrics.metricsContext.isRecoveryStart): and ORM will start reporting this way. You might need to look at the insert of stroke 0 then, triggered by

openrowingmonitor/app/recorders/rowingDataRecorder.js

Line 53 in b380437

case (metrics.metricsContext.isSessionStart):

As that might be injecting the drive anyway.

Would be interesting, as the start behaviour of the PM5 is still a bit of a mystery to me. If we could exclude that from our validation set, stuff like identifying the FlywheelInertia and all other parameters would become a lot simpler.