[Linuxptp-devel] TAI vs UTC confusion

[Libor Pechacek]

Hi,

While playing with a group of hosts using both hardware and software
timestamping I've noticed a discrepancy in UTC/TAI usage.

With master using software timestamping (i.e. using system clock for
timestamps) and slave having a PHC, using hardware timestamping and relying on
phc2sys to sync system clock to PHC, all seems fine until I use phc2sys -w
option. At that point the slave system clock is exactly the 35 seconds behind
master system clock.

Similarly, in reverse setup, master synchronizing system clock to PHC (PHC set
as phc2sys slave here), PHC being broadcast to network by ptp4l, slave using
software timestamping directly synchronizes its system clock to what is coming
from network, the slave is again forced 35 seconds behind the master unless I
use -w on the master.

To sum it up, with the software timestamping master it seems that UTC is
broadcast to the network. I looked into port_tx_sync() and didn't see any
offset calculation for the transmitted timestamp. Didn't dig deeper. Is this
intended or is that a bug?

In the second case, phc2sys by default keeps the same time at both PHC and
system clock, while as to my understanding PHC should be in TAI and system
clock in UTC. Without using -w or -O phc2sys does not maintain any offset
between these two. Shouldn't be -w by default on?

Libor

[Miroslav Lichvar]

Post by Libor Pechacek
Hi,
While playing with a group of hosts using both hardware and software
timestamping I've noticed a discrepancy in UTC/TAI usage.
With master using software timestamping (i.e. using system clock for
timestamps) and slave having a PHC, using hardware timestamping and relying on
phc2sys to sync system clock to PHC, all seems fine until I use phc2sys -w
option. At that point the slave system clock is exactly the 35 seconds behind
master system clock.

This looks like a bug. The currentUtcOffset value should probably be
used only when the PTP_TIMESCALE flag is set. Can you please test it
with this patch?

--- a/phc2sys.c
+++ b/phc2sys.c
@@ -440,8 +440,9 @@ static int run_pmc(struct clock *clock, int timeout,

break;
case TIME_PROPERTIES_DATA_SET:
- clock->sync_offset = ((struct timePropertiesDS *)data)->
- currentUtcOffset;
+ if (((struct timePropertiesDS *)data)->flags & PTP_TIMESCALE)
+ clock->sync_offset = ((struct timePropertiesDS *)data)->
+ currentUtcOffset;
if (((struct timePropertiesDS *)data)->flags & LEAP_61)
clock->leap = 1;
else if (((struct timePropertiesDS *)data)->flags & LEAP_59)

Post by Libor Pechacek
Similarly, in reverse setup, master synchronizing system clock to PHC (PHC set
as phc2sys slave here), PHC being broadcast to network by ptp4l, slave using
software timestamping directly synchronizes its system clock to what is coming
from network, the slave is again forced 35 seconds behind the master unless I
use -w on the master.

I think that is working as expected. Without -w, the phc2sys running
on the PTP master doesn't know the currentUtcOffset value and sets the
PHC from the system clock without any TAI-UTC correction.

[Libor Pechacek]

Hi Miroslav,

Post by Miroslav Lichvar

Post by Libor Pechacek
While playing with a group of hosts using both hardware and software
timestamping I've noticed a discrepancy in UTC/TAI usage.
With master using software timestamping (i.e. using system clock for
timestamps) and slave having a PHC, using hardware timestamping and relying on
phc2sys to sync system clock to PHC, all seems fine until I use phc2sys -w
option. At that point the slave system clock is exactly the 35 seconds behind
master system clock.

This looks like a bug. The currentUtcOffset value should probably be
used only when the PTP_TIMESCALE flag is set. Can you please test it
with this patch?

Thanks for the patch, that indeed fixes the problem.

Post by Miroslav Lichvar
--- a/phc2sys.c
+++ b/phc2sys.c
@@ -440,8 +440,9 @@ static int run_pmc(struct clock *clock, int timeout,
break;
- clock->sync_offset = ((struct timePropertiesDS *)data)->
- currentUtcOffset;
+ if (((struct timePropertiesDS *)data)->flags & PTP_TIMESCALE)
+ clock->sync_offset = ((struct timePropertiesDS *)data)->
+ currentUtcOffset;
if (((struct timePropertiesDS *)data)->flags & LEAP_61)
clock->leap = 1;
else if (((struct timePropertiesDS *)data)->flags & LEAP_59)

Post by Libor Pechacek
Similarly, in reverse setup, master synchronizing system clock to PHC (PHC set
as phc2sys slave here), PHC being broadcast to network by ptp4l, slave using
software timestamping directly synchronizes its system clock to what is coming
from network, the slave is again forced 35 seconds behind the master unless I
use -w on the master.

I think that is working as expected. Without -w, the phc2sys running
on the PTP master doesn't know the currentUtcOffset value and sets the
PHC from the system clock without any TAI-UTC correction.

After having look into the specs, I can see linuxptp does not use PTP
timescale. I think ptp4l relatioship to PTP timescale is worth documenting in
the man page.

Libor

[Richard Cochran]

Post by Libor Pechacek
After having look into the specs, I can see linuxptp does not use PTP
timescale. I think ptp4l relatioship to PTP timescale is worth documenting in
the man page.

Why do you say that it does not use the PTP timescale?

If running as a slave, ptp4l uses what the master has advertised.

If running as a grandmaster, ptp4l uses PTP timescale (for HW time
stamping + PHC) or UTC (for SW time stamping + system clock).

Thanks,
Richard

[Libor Pechacek]

Hello Richard,

Post by Richard Cochran

Post by Libor Pechacek
After having look into the specs, I can see linuxptp does not use PTP
timescale. I think ptp4l relatioship to PTP timescale is worth documenting in
the man page.

Why do you say that it does not use the PTP timescale?
If running as a slave, ptp4l uses what the master has advertised.

Indeed. After having another look at the sources I can see how is
PTP_TIMESCALE flag processed by slave.

Post by Richard Cochran
If running as a grandmaster, ptp4l uses PTP timescale (for HW time
stamping + PHC) or UTC (for SW time stamping + system clock).

Thanks for clarification. I overlooked utc_timescale handling in clock_create.
Therefore ptp4l is consistent in its behavior from my point of view.

Yet, there is a pitfall left with phc2sys. Consider following example:
master syncing system clock to PHC using phc2sys with *zero* offset (default)
ptp4l on the master uses hardware timestamping and advertises use of PTP
timescale, however the PHC is running in UTC due to the above zero offset
slave can see ptpTimescale == 1 and applies UTC/TAI correction
As a result slave is 35 seconds behind master

Given that the offset between system clock and PTP hardware clock is
configurable with significant impact, I think the use of timescales and offsets
deserves documenting in the manual page if we don't come up with a robust
offset handling in phc2sys.

Libor


Data from testing to illustrate the above example (fux is master, liszt is
slave).
liszt:~/:[0]# ps $(pidof ptp4l)
PID TTY STAT TIME COMMAND
14115 pts/1 S 0:00 /usr/sbin/ptp4l -f /etc/ptp4l.conf -i eth0 -2 -S
liszt:~/:[0]# pmc -2 'GET TIME_PROPERTIES_DATA_SET';
sending: GET TIME_PROPERTIES_DATA_SET
003048.fffe.c9bdf2-1 seq 0 RESPONSE MANAGMENT TIME_PROPERTIES_DATA_SET
currentUtcOffset 35
leap61 0
leap59 0
currentUtcOffsetValid 0
ptpTimescale 1
timeTraceable 0
frequencyTraceable 0
timeSource 0xa0
liszt:~/:[0]# ssh fux 'echo -n master:\ ; date'; echo -n slave:\ ; date;
master: Fri May 3 10:10:00 CEST 2013
slave: Fri May 3 10:09:25 CEST 2013
liszt:~/:[0]# tail -3 /var/log/messages;
May 3 10:09:22 liszt ptp4l: [357993.599] master offset 80734 s2 freq +151151 path delay 91388
May 3 10:09:23 liszt ptp4l: [357994.599] master offset -10195 s2 freq +142048 path delay 74902
May 3 10:09:24 liszt ptp4l: [357995.599] master offset -8661 s2 freq +142193 path delay 74902
liszt:~/:[0]# ssh fux 'ps $(pidof ptp4l) $(pidof phc2sys)'
PID TTY STAT TIME COMMAND
3788 pts/1 S 0:08 /usr/sbin/ptp4l -f /etc/ptp4l.conf -i eth0 -2
4890 pts/1 S 0:00 /usr/sbin/phc2sys -c /dev/ptp0 -s CLOCK_REALTIME

[Miroslav Lichvar]

Post by Libor Pechacek
Given that the offset between system clock and PTP hardware clock is
configurable with significant impact, I think the use of timescales and offsets
deserves documenting in the manual page if we don't come up with a robust
offset handling in phc2sys.

It would be nice to have it described in the phc2sys man page.
Also, an EXAMPLES section showing the common use cases could be
useful.

[Libor Pechacek]

Post by Miroslav Lichvar

Post by Libor Pechacek
Given that the offset between system clock and PTP hardware clock is
configurable with significant impact, I think the use of timescales and offsets
deserves documenting in the manual page if we don't come up with a robust
offset handling in phc2sys.

It would be nice to have it described in the phc2sys man page.
Also, an EXAMPLES section showing the common use cases could be
useful.

That's my aim.

Yet, I've spent some more cycles thinking about the desired behavior and
staring into the code.

My assumptions:
* UTC/TAI offset should be kept at one place, ideally ptp4l
* PHC should run TAI whenever possible, CLOCK_REALTIME is UTC

With that I think getting UTC offset from ptp4l via PMC would be sensible in
cases when user does not specify offset on command line. That would mean a
code change similar to the following:

diff --git a/phc2sys.c b/phc2sys.c
index e5d34bb..2aedf5a 100644
--- a/phc2sys.c
+++ b/phc2sys.c
@@ -684,7 +689,6 @@ int main(int argc, char *argv[])
print_set_syslog(use_syslog);
print_set_level(print_level);

- if (wait_sync) {
if (init_pmc(&dst_clock, domain_number))
return -1;

@@ -712,7 +716,6 @@ int main(int argc, char *argv[])

if (forced_sync_offset || !dst_clock.sync_offset_direction)
close_pmc(&dst_clock);
- }

ppb = clockadj_get_freq(dst_clock.clkid);
/* The reading may silently fail and return 0, reset the frequency to

Comments, objections?

Libor

[Miroslav Lichvar]

Post by Libor Pechacek
With that I think getting UTC offset from ptp4l via PMC would be sensible in
cases when user does not specify offset on command line. That would mean a
diff --git a/phc2sys.c b/phc2sys.c
index e5d34bb..2aedf5a 100644
--- a/phc2sys.c
+++ b/phc2sys.c
@@ -684,7 +689,6 @@ int main(int argc, char *argv[])
print_set_syslog(use_syslog);
print_set_level(print_level);
- if (wait_sync) {
if (init_pmc(&dst_clock, domain_number))
return -1;

This change would make phc2sys always wait for ptp4l, I'm not sure
that's what we want.

I think the valid use cases are:
- wait for ptp4l, determine the offset automatically
- wait for ptp4l, set the offset with -O
- don't wait for ptp4l, set the offset with -O

The -O option alone is not enough to switch between the three cases.
Would it make sense if phc2sys required that at least one of the -w,
-O is used, i.e. don't have a default value for -O?

Thanks,

[Libor Pechacek]

Post by Miroslav Lichvar

Post by Libor Pechacek
With that I think getting UTC offset from ptp4l via PMC would be sensible in
cases when user does not specify offset on command line. That would mean a
diff --git a/phc2sys.c b/phc2sys.c
index e5d34bb..2aedf5a 100644
--- a/phc2sys.c
+++ b/phc2sys.c
@@ -684,7 +689,6 @@ int main(int argc, char *argv[])
print_set_syslog(use_syslog);
print_set_level(print_level);
- if (wait_sync) {
if (init_pmc(&dst_clock, domain_number))
return -1;

This change would make phc2sys always wait for ptp4l, I'm not sure
that's what we want.

Well, depend on what means "wait". Phc2sys does wait for ptp4l to be started
to offer the socket but does not wait for the port state change (sync).

Post by Miroslav Lichvar
- wait for ptp4l, determine the offset automatically
- wait for ptp4l, set the offset with -O
- don't wait for ptp4l, set the offset with -O

- wait for ptp4l to be started to read the UTC offset from it

I accept that phc2sys cannot be used standalone then, which I don't like.

Post by Miroslav Lichvar
The -O option alone is not enough to switch between the three cases.
Would it make sense if phc2sys required that at least one of the -w,
-O is used, i.e. don't have a default value for -O?

Yes, that fits my idea of acceptable default. Thanks!

Libor

[Miroslav Lichvar]

Post by Libor Pechacek

Post by Miroslav Lichvar
This change would make phc2sys always wait for ptp4l, I'm not sure
that's what we want.

Well, depend on what means "wait". Phc2sys does wait for ptp4l to be started
to offer the socket but does not wait for the port state change (sync).

Hm, with -w it should wait for ptp4l to have at least one port in the
master or slave state and then fetch timePropertiesDS for the offset.

Does it not work as intended?

Thanks,

[Libor Pechacek]

Post by Miroslav Lichvar

Post by Libor Pechacek

Post by Miroslav Lichvar
This change would make phc2sys always wait for ptp4l, I'm not sure
that's what we want.

Well, depend on what means "wait". Phc2sys does wait for ptp4l to be started
to offer the socket but does not wait for the port state change (sync).

Hm, with -w it should wait for ptp4l to have at least one port in the
master or slave state and then fetch timePropertiesDS for the offset.

Yes, with -w phc2sys waits for ptp4l to have a port in slave or master mode.

My code change makes phc2sys to read UTC offset from ptp4l when neither -O nor
-w is specified. That causes phc2sys to issue management request to read
TIMESCALE_PROPERTIES_DATASET to get the offset and then proceed with the sync
loop.

Recap of phc2sys behavior after my code change. Assume PTP timescale in use.

- neither -O nor -w specified
- phc2sys reads UTC offset from TIMESCALE_PROPERTIES_DATASET, does not wait
for a port state change
- -O specified
- uset offset from -O, does not wait for a port state change
- -w specified
- read UTC offset from TIMESCALE_PROPERTIES_DATASET and wait for a port to
become master or slave
- both -w and -O specified
- use UTC offset from -O and wait for a port to become master or slave

The new limitation is that if ptp4l is not running, phc2sys is waiting for the
communication socket to appear. That would be acceptable if we declared
phc2sys to be run always with ptp4l.

Post by Miroslav Lichvar
Does it not work as intended?

All works fine. ;)

Libor

[Keller, Jacob E]

-----Original Message-----
Sent: Friday, May 10, 2013 1:05 AM
To: Miroslav Lichvar
Subject: Re: [Linuxptp-devel] TAI vs UTC confusion

Post by Miroslav Lichvar

Post by Libor Pechacek

Post by Miroslav Lichvar
This change would make phc2sys always wait for ptp4l, I'm not sure
that's what we want.

Well, depend on what means "wait". Phc2sys does wait for ptp4l to

be started

Post by Miroslav Lichvar

Post by Libor Pechacek
to offer the socket but does not wait for the port state change (sync).

Hm, with -w it should wait for ptp4l to have at least one port in the
master or slave state and then fetch timePropertiesDS for the offset.

Yes, with -w phc2sys waits for ptp4l to have a port in slave or master mode.
My code change makes phc2sys to read UTC offset from ptp4l when neither -O nor
-w is specified. That causes phc2sys to issue management request to read
TIMESCALE_PROPERTIES_DATASET to get the offset and then proceed with the sync
loop.
Recap of phc2sys behavior after my code change. Assume PTP timescale in use.
- neither -O nor -w specified
- phc2sys reads UTC offset from TIMESCALE_PROPERTIES_DATASET, does not wait
for a port state change
- -O specified
- uset offset from -O, does not wait for a port state change
- -w specified
- read UTC offset from TIMESCALE_PROPERTIES_DATASET and wait for a port to
become master or slave
- both -w and -O specified
- use UTC offset from -O and wait for a port to become master or slave
The new limitation is that if ptp4l is not running, phc2sys is waiting for the
communication socket to appear. That would be acceptable if we declared
phc2sys to be run always with ptp4l.

Post by Miroslav Lichvar
Does it not work as intended?

All works fine. ;)
Libor

I'm not really sure we want to enforce that, though?

- Jake