You can probably just download and use the latest driver from:
back-ported that would change the APIs.
Post by Mattias BarthelThanks a lot Alex, there might still be some hope then.
Unfortunately Im bound to using the linux kernel 3.0.8. What
out-of-tree driver can I use with that, please?
On Tuesday, October 13, 2015, Alexander Duyck
I don't know if it is available in that driver or not, but there
is an option in later igb drivers to support a time stamp being
written with the packet data. Based on the fact that you are
already seeing issues, you might want to try getting the
out-of-tree driver from e1000.sf.net <http://e1000.sf.net> and
seeing if you can use that with the time stamp in packet option
that is enabled when you select to timestamp all incoming Rx packets.
- Alex
Yes, setting the cycles.shift to the same as on the 82580 and
also shifting the reads accordingly everything works. The sync
of shhwtstamps->syststamp too.
My driver version is 3.0.6-k2 and linux is 3.0.8.
Although this might not be very suited to my needs.
The incoming buffers wont be timestamped if you cant keep up
with reading the timestamping registers as I understand it.
This might be perfectly fine in the case of PTP where the
packet rate is fairly low. In my case Im looking at some
80kpps that all would need to be timestamped.
*//* If this bit is set, then the RX registers contain the
time stamp. No/**/ other packet will be time stamped until we
read these registers, so/* */read the registers to make them
available again. Because only one/**/ packet can be time
stamped at a time, we know that the register/* */values must
belong to this one here and therefore we don't need to/**/
compare any of the additional attributes stored for it./* */If
nothing went wrong, then it should have a shared tx_flags that
we/**/ can turn into a skb_shared_hwtstamps. *//*
*//*
*//*
On Tuesday, October 13, 2015, Alexander Duyck
I'm assuming you are using either an older kernel or an
out-of-tree driver do I have that right? I ask because this code
doesn't resemble what is is currently in the latest Linux kernel.
Comments on what you have found are inline below.
- Alex
Hi, I think I managed to get around this problem.
----
/*
* The 82580 timesync updates the system timer every 8ns by 8ns
* and the value cannot be shifted. Instead we need to shift
* the registers to generate a 64bit timer value. As a result
* SYSTIMR/L/H, TXSTMPL/H, RXSTMPL/H all have to be shifted by
* 24 in order to generate a larger value for
synchronization.
*/
adapter->cycles.shift = IGB_82580_TSYNC_SHIFT;
----
I have the I350 and I dont think this cycles shift was meant
for my adapter.
IGB_82580_TSYNC_SHIFT is 24.
Also when doing igb_read_clock,
this shifting is applied only for the 82580 and I guess it
--------
/*
* The timestamp latches on lowest register read. For the 82580
* the lowest register is SYSTIMR instead of SYSTIML.
However
we never
* adjusted TIMINCA so SYSTIMR will just read as all 0s so
ignore it.
*/
if (hw->mac.type == e1000_82580) {
stamp = rd32(E1000_SYSTIMR) >> 8;
shift = IGB_82580_TSYNC_SHIFT;
}
stamp |= (u64)rd32(E1000_SYSTIML) << shift;
stamp |= (u64)rd32(E1000_SYSTIMH) << (shift + 32);
-------------
This is a bug in the code. The i350 should be shifted by the same
value as the 82580. Then the result would be the same as if you
had set the shift to 0. As I recall the reasoning behind the
change was to make use of as many bits of the time as possible,
and on 82580 and newer the SYSTIMH portion of the clock only
contained 8 bits of actual clock data.
Without that change I believe there is a risk of the counter being
considered to have cycled too fast and causing errors as the cycle
counter will roll over at 40 bits instead of 64. The alternative
would be to update the cycle counter you are using so that it is
aware that it is limited to 40 bits and then use an shift of 0.
Setting adapter->cycles.shift to 0 makes the clock run at
about the same speed as the system clock.
Next problem I have is that the syststamp is not syncing
correctly against the system clock.
Any help is much appreciated.
Regards,
Mattias
On Mon, Oct 12, 2015 at 5:04 PM, Mattias Barthel
Hi again,
Started to try and timestamp all packets in the I350.
driver: igb
version: 3.0.6-k2
firmware-version: 0.147-0
Doing an ioctl that turns on timestamping for all packets.
hwconfig.tx_type = HWTSTAMP_TX_OFF;
hwconfig.rx_filter = HWTSTAMP_FILTER_ALL;
hwconfig_requested = hwconfig;
ioctl(sock, SIOCSHWTSTAMP, &hwtstamp)
The strangest thing is that although it works to read the
timestamps the clock seems to advance very slowly
and/or
incorrectly.
I just printout the HW timestamps alongside the
systemclock and
the hw stamping is advancing the nanoseconds when the
systemclock
is advancing us and seconds.
Is there a know bug in this version of the driver that maybe
initializes the clock incorrectly or maybe reads
incorrectly from
the clock?
[ 63.004116] igb_rx_hwtstamp
1444656871372845807
[ 63.005721] system timestamp: tv_sec: 1444656793,
tv_usec: 683825
1444656871372845807,
tv_sec: 1444656871, tv_usec: 372845
1444656871372825302,
1444656871, tv_usec: 372825
[ 63.009395]
[ 63.104111] igb_rx_hwtstamp
1444656871372845813
[ 63.105747] system timestamp: tv_sec: 1444656793,
tv_usec: 783851
1444656871372845813,
tv_sec: 1444656871, tv_usec: 372845
1444656871372825308,
1444656871, tv_usec: 372825
[ 63.109428]
[ 63.204109] igb_rx_hwtstamp
1444656871372845819
[ 63.206018] system timestamp: tv_sec: 1444656793,
tv_usec: 884122
1444656871372845819,
tv_sec: 1444656871, tv_usec: 372845
1444656871372825314,
1444656871, tv_usec: 372825
[ 63.210897]
[ 63.304068] igb_rx_hwtstamp
1444656871372845825
[ 63.306176] system timestamp: tv_sec: 1444656793,
tv_usec: 984280
1444656871372845825,
tv_sec: 1444656871, tv_usec: 372845
1444656871372825320,
1444656871, tv_usec: 372825
[ 63.311032]
[ 63.404107] igb_rx_hwtstamp
1444656871372845831
[ 63.405695] system timestamp: tv_sec: 1444656794,
tv_usec: 83798
1444656871372845831,
tv_sec: 1444656871, tv_usec: 372845
1444656871372825326,
1444656871, tv_usec: 372825
[ 63.409405]
[ 63.504084] igb_rx_hwtstamp
1444656871372845837
[ 63.505715] system timestamp: tv_sec: 1444656794,
tv_usec: 183819
1444656871372845837,
tv_sec: 1444656871, tv_usec: 372845
1444656871372825332,
1444656871, tv_usec: 372825
Any help will be very much appreciated.
Best regards,
Mattias
On Thu, Oct 8, 2015 at 6:01 PM, Alexander Duyck
The i350 should support timestamping all packets as I
recall. You should probably look into getting the
linuxptp package and
seeing if you can make use of the hwstamp_ctl command to
enable timestamping for all incoming packets
on the
adapter.
- Alex
Hi
thanks for answer.
Unfortunately, the udp/ip stream just as
the cfm
stream is
against only one peer.
Furthemore on the guest side I have only
one vcpu
tied to
two physical siblings to minimize L2 cache
misses
on ISRs
due to cache coherence.
I had an idea this morning, maybe I can
make use
of the
ptp hw timestaming for this? This should give
better accuracy.
Is this possible or does it only work for
ethertype ptp?
On Thursday, October 8, 2015, Alexander Duyck
The difference could be Receive Side Scaling
(RSS). The UDP/IP
packets
can be load balanced across all queues
via RSS
based
on a hash of the
source and destination addresses. So
if you have
multiple sources
generating these packets, or they are
being
received
on multiple IP
addresses then it is possible the load is
being spread
out over
multiple
CPUs. Your CFM packets are on an Ethertype
other than
IPv4 or IPv6 so
all packets will end up on queue 0 if
I am not
mistaken.
- Alex
Post by Mattias BarthelHi again!
I realise my message might not have
been so
clear.
Post by Mattias BarthelI will try to clarify. I am doing PM
with
software
timestamping.
Post by Mattias Barthelversion: 3.0.6-k2
firmware-version: 0.147-0
I do timestamping (NTP format) of the
packets in the
same place
for all
Post by Mattias Barthelprotocols. (in igb_clean_rx_irq_adv()).
For these non ip packets like IEEE
801.2ag CFM ETH-DM
(ethertype 0x8902) I
Post by Mattias Barthelget quite a lot worse accuracy in the
timestamping.
Post by Mattias BarthelFor UDP/IP TWAMP I get for 10k pps a max
delay of
around 140us
back to back.
Post by Mattias BarthelFor CFM ETH DM I get for the same packet
rate and
packet size a
max delay
Post by Mattias Barthelof around 900us.
Could FW be putting these L2 packets
in another
queue with different
Post by Mattias Barthelcharacteristics?
I tried to add a ET-type filter as
written in my
previous mail
but it
Post by Mattias Barthelshowed no difference.
Thanks for any help given!
On Mon, Oct 5, 2015 at 2:19 PM,
Mattias Barthel
<javascript:;>>
using the igb
driver on i350
on linux
Post by Mattias BarthelPost by Mattias Barthelregarding
ETH CFM packets. This compared to TWAMP
(IPv4 UDP
packets).
i350 devices in
PCI-passthrough.
(filter) for
those types of
ethernet
Post by Mattias BarthelPost by Mattias Barthelprotocol packets.
wr32(E1000_ETQF(4),
(1 << 31) | /* queue enable */
(E1000_ETQF_FILTER_ENABLE) | /* enable
filter */
interrupt */
Post by Mattias BarthelPost by Mattias Barthel(0x4 << 16) | /* queue no. 4 */
(ETH_P_8021AG)); /* 0x8902
CFM eth
protocol type */
Post by Mattias BarthelPost by Mattias BarthelETH_P_8021A is 0x8902
Is this a correct/good approach?
Thanks in advance!
--
Mattias Barthel
------------------------------------------------------------------------------
_______________________________________________
E1000-devel mailing list
<javascript:;>
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
Avis de confidentialité
Les informations contenues dans le présent
message
et dans
toute pièce qui lui est jointe sont confidentielles et
peuvent être protégées par le secret
professionnel. Ces
informations sont à l’usage exclusif de
son ou de ses
destinataires. Si vous recevez ce message par erreur,
veuillez s’il vous plait communiquer
immédiatement
avec
l’expéditeur et en détruire tout
exemplaire. De
plus, il
vous est strictement interdit de le divulguer, de le
distribuer ou de le reproduire sans l’autorisation de
l’expéditeur. Merci.
Confidentiality notice
This e-mail message and any attachment hereto contain
confidential information which may be
privileged
and which
is intended for the exclusive use of its
addressee(s). If
you receive this message in error, please
inform
sender
immediately and destroy any copy thereof.
Furthermore, any
disclosure, distribution or copying of this
message and/or
any attachment hereto without the consent
of the
sender is
strictly prohibited. Thank you.
-- Mattias Barthel
-- Mattias Barthel
Avis de confidentialité
Les informations contenues dans le présent message et dans
toute pièce qui lui est jointe sont confidentielles et peuvent
être protégées par le secret professionnel. Ces informations
sont à l’usage exclusif de son ou de ses destinataires. Si
vous recevez ce message par erreur, veuillez s’il vous plait
communiquer immédiatement avec l’expéditeur et en détruire
tout exemplaire. De plus, il vous est strictement interdit de
le divulguer, de le distribuer ou de le reproduire sans
l’autorisation de l’expéditeur. Merci.
Confidentiality notice
This e-mail message and any attachment hereto contain
confidential information which may be privileged and which is
intended for the exclusive use of its addressee(s). If you
receive this message in error, please inform sender
immediately and destroy any copy thereof. Furthermore, any
disclosure, distribution or copying of this message and/or any
attachment hereto without the consent of the sender is
strictly prohibited. Thank you.
Avis de confidentialité
Les informations contenues dans le présent message et dans toute pièce
qui lui est jointe sont confidentielles et peuvent être protégées par
le secret professionnel. Ces informations sont à l’usage exclusif de
son ou de ses destinataires. Si vous recevez ce message par erreur,
veuillez s’il vous plait communiquer immédiatement avec l’expéditeur
et en détruire tout exemplaire. De plus, il vous est strictement
interdit de le divulguer, de le distribuer ou de le reproduire sans
l’autorisation de l’expéditeur. Merci.
Confidentiality notice
This e-mail message and any attachment hereto contain confidential
information which may be privileged and which is intended for the
exclusive use of its addressee(s). If you receive this message in
error, please inform sender immediately and destroy any copy thereof.
Furthermore, any disclosure, distribution or copying of this message
and/or any attachment hereto without the consent of the sender is
strictly prohibited. Thank you.