Sample Header Ad - 728x90

Debugging random loss of communication in embedded Linux device using PPP and cellular connection

0 votes
1 answer
108 views
I am working on setting up an embedded linux device so that it can connect to a backend server via cellular connection. I am using PPP version 2.4.2 and a Sequans cell chip. I have the device working and talking to the backend. To test its performance, the backend will send a message every 5 minutes, and the device should respond. At random intervals for random amounts of time, my device seems to lose connection and will randomly come back. The cell chip does not lose connection, and ppp does not go down. Everything seems fine within my application logs. To debug this issue, I have been using ppp's **record** ***filename*** option and tcpdump on my ppp interface. What I have found is the during this loss of communication, tcpdump shows that nothing is coming in or out of the ppp interface. The ppp record file shows that *something* is happening within ppp. Does anyone know what is going on within ppp during this time? The snippet below is from the file (which I opened using pppdump -h *filename*). It seems like what is being sent out is prepending a c0 at the beginning when it gets in this weird sent/rcvd loop. In some cases, communication continues on its own. In other cases, the device will send a modem ping to the backend which will then cause the device to receive all the messages that it missed from the backend and responds to all of them. The snippet shows normal communication, followed by the sent/rcvd loop, followed by the message finally being sent. I have put XX over the part that would have been the IP address.
rcvd   7e 21 45 00 00 23 72 91 00 00 7c 11 36 d5 XX XX  ~!E..#r...|.6..(
       XX XX XX XX XX XX 04 64 c7 7a 00 0f 5a d2 7d 5d   _.,`..d.z..Z.}]
rcvd   04 00 50 06 66 c0 15 c2 7e                       ..P.f...~
time  8.0s
sent   7e 21 45 00 00 30 b1 0a 40 00 40 11 f4 4e XX XX  ~!E..0..@.@..N.,
       XX XX XX XX XX XX c7 7a 04 64 00 1c d1 3c 7d 5d  `..( _.z.d...<}]
       11 a0 67 2c 11 f5 38 dd 45 34 bb 21 b8 88 bc 9e  ..g,..8.E4.!....
       f5 33 08 b0 a3 7e                                .3...~
time  4.5s
rcvd   7e 21 45 00 00 23 72 92 00 00 7c 11 36 d4 XX XX  ~!E..#r...|.6..(
       XX XX XX XX XX XX 04 64 c7 7a 00 0f 59 d1 7d 5d   _.,`..d.z..Y.}]
rcvd   04 00 50 06 67 c1 c2 ac 7e                       ..P.g...~
time  0.1s
sent   7e 21 45 00 00 30 b2 aa 40 00 40 11 f2 ae XX XX ~!E..0..@.@....,
       XX XX XX XX XX XX c7 7a 04 64 00 1c 8f 99 7d 5d  `..( _.z.d....}]
       11 a0 68 c5 60 17 31 07 2d 07 56 ff 14 63 3c bf  ..h.`.1.-.V..c<.
       6b e4 8d cc 58 7e                                k...X~
time  0.2s
rcvd   7e 21 45 00 00 23 72 93 00 00 7c 11 36 d3 XX XX ~!E..#r...|.6..(
       XX XX XX XX XX XX 04 64 c7 7a 00 0f 58 d0 7d 5d   _.,`..d.z..X.}]
rcvd   04 00 50 06 68 c2 ed 79 7e                       ..P.h..y~
time  4.3s
sent   7e c0 21 09 59 00 08 83 0d 23 8a 31 3a 7e        ~.!.Y....#.1:~
rcvd   7e ff 03 c0 21 0a 59 00 08 2b b6 cb 61 cd 98 7e  ~...!.Y..+..a..~
time  30.1s
sent   7e c0 21 09 5a 00 08 83 0d 23 8a 5f 92 7e        ~.!.Z....#._.~
rcvd   7e ff 03 c0 21 0a 5a 00 08 2b b6 cb 61 a3 30 7e  ~...!.Z..+..a.0~
time  30.0s
sent   7e c0 21 09 5b 00 08 83 0d 23 8a 8a 0d 7e        ~.!.[....#...~
rcvd   7e ff 03 c0 21 0a 5b 00 08 2b b6 cb 61 76 af 7e  ~...!.[..+..av.~
time  30.0s
sent   7e c0 21 09 5c 00 08 83 0d 23 8a 92 ca 7e        ~.!.\....#...~
rcvd   7e ff 03 c0 21 0a 5c 00 08 2b b6 cb 61 6e 68 7e  ~...!.\..+..anh~
time  30.0s
sent   7e c0 21 09 5d 00 08 83 0d 23 8a 47 55 7e        ~.!.]....#.GU~
rcvd   7e ff 03 c0 21 0a 5d 00 08 2b b6 cb 61 bb f7 7e  ~...!.]..+..a..~
time  30.1s
sent   7e c0 21 09 5e 00 08 83 0d 23 8a 29 fd 7e        ~.!.^....#.).~
rcvd   7e ff 03 c0 21 0a 5e 00 08 2b b6 cb 61 d5 5f 7e  ~...!.^..+..a._~
time  30.0s
sent   7e c0 21 09 5f 00 08 83 0d 23 8a fc 62 7e        ~.!._....#..b~
rcvd   7e ff 03 c0 21 0a 5f 00 08 2b b6 cb 61 00 c0 7e  ~...!._..+..a..~
time  30.0s
sent   7e c0 21 09 60 00 08 83 0d 23 8a 42 ad 7e        ~.!.`....#.B.~
rcvd   7e ff 03 c0 21 0a 60 00 08 2b b6 cb 61 be 0f 7e  ~...!.`..+..a..~
time  30.0s
sent   7e c0 21 09 61 00 08 83 0d 23 8a 97 32 7e        ~.!.a....#..2~
rcvd   7e ff 03 c0 21 0a 61 00 08 2b b6 cb 61 6b 90 7e  ~...!.a..+..ak.~
time  30.1s
sent   7e c0 21 09 62 00 08 83 0d 23 8a f9 9a 7e        ~.!.b....#...~
rcvd   7e ff 03 c0 21 0a 62 00 08 2b b6 cb 61 05 38 7e  ~...!.b..+..a.8~
time  30.0s
sent   7e c0 21 09 63 00 08 83 0d 23 8a 2c 05 7e        ~.!.c....#.,.~
rcvd   7e ff 03 c0 21 0a 63 00 08 2b b6 cb 61 d0 a7 7e  ~...!.c..+..a..~
time  30.0s
sent   7e c0 21 09 64 00 08 83 0d 23 8a 34 c2 7e        ~.!.d....#.4.~
rcvd   7e ff 03 c0 21 0a 64 00 08 2b b6 cb 61 c8 60 7e  ~...!.d..+..a.`~
time  30.1s
sent   7e c0 21 09 65 00 08 83 0d 23 8a e1 5d 7e        ~.!.e....#..]~
rcvd   7e ff 03 c0 21 0a 65 00 08 2b b6 cb 61 1d ff 7e  ~...!.e..+..a..~
time  30.0s
sent   7e c0 21 09 66 00 08 83 0d 23 8a 8f f5 7e        ~.!.f....#...~
rcvd   7e ff 03 c0 21 0a 66 00 08 2b b6 cb 61 73 57 7e  ~...!.f..+..asW~
time  30.0s
sent   7e c0 21 09 67 00 08 83 0d 23 8a 5a 6a 7e        ~.!.g....#.Zj~
rcvd   7e ff 03 c0 21 0a 67 00 08 2b b6 cb 61 a6 c8 7e  ~...!.g..+..a..~
time  30.0s
sent   7e c0 21 09 68 00 08 83 0d 23 8a ae 73 7e        ~.!.h....#..s~
rcvd   7e ff 03 c0 21 0a 68 00 08 2b b6 cb 61 52 d1 7e  ~...!.h..+..aR.~
time  30.1s
sent   7e c0 21 09 69 00 08 83 0d 23 8a 7b ec 7e        ~.!.i....#.{.~
rcvd   7e ff 03 c0 21 0a 69 00 08 2b b6 cb 61 87 4e 7e  ~...!.i..+..a.N~
time  28.8s
sent   7e 21 45 00 00 3b 10 38 40 00 40 11 95 16 XX XX ~!E..;.8@.@....,
       XX XX XX XX XX XX c7 7a 04 64 00 27 ed 37 7d 5d  `..( _.z.d.'.7}]
       1c 00 55 03 fe 01 30 31 35 37 37 30 30 30 31 38  ..U...0157700018
       32 32 33 38 37 f7 8e 68 c0 df 78 5b 53 28 5c 74  22387..h..x[S(\t
       7e                                               ~
sent   21 45 00 00 30 10 39 40 00 40 11 95 20 XX XX XX !E..0.9@.@.. .,`
       XX XX XX XX XX c7 7a 04 64 00 1c b0 64 7d 5d 11  ..( _.z.d...d}].
       a0 69 04 ee 90 77 36 49 41 d0 ce 5f 9b 2f 88 4f  .i...w6IA.._./.O
       d1 35 c1 11 7e                                   .5..~
time  1.2s
sent   7e c0 21 09 6a 00 08 83 0d 23 8a 15 44 7e        ~.!.j....#..D~
rcvd   7e ff 03 c0 21 0a 6a 00 08 2b b6 cb 61 e9 e6 7e  ~...!.j..+..a..~
time  3.8s
sent   7e 21 45 00 00 30 11 f7 40 00 40 11 93 62 XX XX ~!E..0..@.@..b.,
       XX XX XX XX XX XX c7 7a 04 64 00 1c b5 e8 7d 5d  `..( _.z.d....}]
       11 a0 69 f1 79 85 ad e8 d8 66 04 71 7d 5e 29 6d  ..i.y....f.q}^)m
       94 4e d5 d0 05 ef 7e                             .N....~
time  2.9s
Asked by jroLayz (1 rep)
Apr 27, 2023, 04:02 PM
Last activity: Apr 28, 2023, 06:14 AM