Working on the Ritchie market data server, and I've hit a UDP socket programming problem. Why is this unicast not being received by the client? I've been hammering away at this for hours. Grr.. ;( - I'm sending price ticks in the form nn.dd; e.g., 42.23, 93.75. Just 5 bytes for test purposes. - The server's (tick generator) IP address is 192.168.1.101 - The client's address is 192.168.1.102 - The return value of the ``sendto`` command is 5, as it should be. - I can successfully ping in both directions: client from server, and server from client; about a 3ms response time. - Both machines are on the same network. - The prices are being generated and sent as shown below - A client process **on the same machine**, (with same IP address of 192.168.1.101) in a different terminal, works fine, as shown in snippet 2 below. - The complete code is included in snippet 3 and 4 - Linksys WRT54GL v1.1 with stock BIOS - This code is a modified version of Beej's UDP example here: http://beej.us/guide/bgnet/output/html/multipage/clientserver.html#datagram - NOTE: Start server with ``tickGenerator 192.168.1.101 helloWorld`` (of course use your IP address(es)). The "helloWorld" from Beej's example will be ignored; I've hacked in a random price generator. - Also posted here on SO https://stackoverflow.com/questions...-client?noredirect=1#comment75643874_44317797 Separate machine at 192.168.1.102 just hangs... Tick generator output: properly generating tick data from 192.168.1.101: Code: Price unicasted: 49.58 Price unicasted: 50.00 Price unicasted: 50.24 ... Client in separate terminal window **on the same machine** (also from from 192.168.1.101) properly receives multicast: Code: listener: waiting to recvfrom... listener: got packet from 192.168.1.101 listener: packet is 5 bytes long listener: packet contains "49.58" listener: got packet from 192.168.1.101 listener: packet is 5 bytes long listener: packet contains "50.00" listener: got packet from 192.168.1.101 listener: packet is 5 bytes long listener: packet contains "50.24" ... Tick generator: Code: /* ** talker.c -- a datagram "client" demo */ #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <errno.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <netdb.h> #include <time.h> #include <stdlib.h> #include <math.h> #include <chrono> #include <thread> #include <iostream> #define SERVERPORT "4950" // the port users will be connecting to void reverse(char *str, int len) { int i=0, j=len-1, temp; while (i<j) { temp = str[i]; str[i] = str[j]; str[j] = temp; i++; j--; } } int intToStr(int x, char str[], int d) { int i = 0; while (x) { str[i++] = (x%10) + '0'; x = x/10; } while (i < d) str[i++] = '0'; reverse(str, i); str[i] = '\0'; return i; } void ftoa(float n, char *res, int afterpoint) { int ipart = (int)n; float fpart = n - (float)ipart; int i = intToStr(ipart, res, 0); if (afterpoint != 0) { res[i] = '.'; fpart = fpart * 100; intToStr((int)fpart, res + i + 1, afterpoint); } } float randPrice() { int b; float d; b = 4950 + rand() % 100 + 1; d = (float)b/100; return d; } int main(int argc, char *argv[]) { int sockfd; struct addrinfo hints, *servinfo, *p; int rv; int numbytes; if (argc != 3) { fprintf(stderr,"usage: talker hostname message\n"); exit(1); } memset(&hints, 0, sizeof hints); hints.ai_family = AF_UNSPEC; hints.ai_socktype = SOCK_DGRAM; if ((rv = getaddrinfo(argv[1], SERVERPORT, &hints, &servinfo)) != 0) { fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv)); return 1; } // loop through all the results and make a socket for(p = servinfo; p != NULL; p = p->ai_next) { if ((sockfd = socket(p->ai_family, p->ai_socktype, p->ai_protocol)) == -1) { perror("talker: socket"); continue; } break; } if (p == NULL) { fprintf(stderr, "talker: failed to create socket\n"); return 2; } char myPrice[6]; float f; while(1) { f = randPrice(); ftoa(f, myPrice, 2); // Convert price to string std::cout << "Price multicasted: " << myPrice << std::endl; if ((numbytes = sendto(sockfd, myPrice, strlen(myPrice), 0, p->ai_addr, p->ai_addrlen)) == -1) { perror("talker: sendto"); exit(1); } std::this_thread::sleep_for(std::chrono::milliseconds(1000)); } freeaddrinfo(servinfo); printf("talker: sent %d bytes to %s\n", numbytes, argv[1]); close(sockfd); return 0; } Tick listener: Code: /* ** listener.c -- a datagram sockets "server" demo */ #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <errno.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <netdb.h> #define MYPORT "4950" // the port users will be connecting to #define MAXBUFLEN 100 // get sockaddr, IPv4 or IPv6: void *get_in_addr(struct sockaddr *sa) { if (sa->sa_family == AF_INET) { return &(((struct sockaddr_in*)sa)->sin_addr); } return &(((struct sockaddr_in6*)sa)->sin6_addr); } int main(void) { int sockfd; struct addrinfo hints, *servinfo, *p; int rv; int numbytes; struct sockaddr_storage their_addr; char buf[MAXBUFLEN]; socklen_t addr_len; char s[INET6_ADDRSTRLEN]; memset(&hints, 0, sizeof hints); hints.ai_family = AF_UNSPEC; // set to AF_INET to force IPv4 hints.ai_socktype = SOCK_DGRAM; hints.ai_flags = AI_PASSIVE; // use my IP if ((rv = getaddrinfo(NULL, MYPORT, &hints, &servinfo)) != 0) { fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv)); return 1; } // loop through all the results and bind to the first we can for(p = servinfo; p != NULL; p = p->ai_next) { if ((sockfd = socket(p->ai_family, p->ai_socktype, p->ai_protocol)) == -1) { perror("listener: socket"); continue; } if (bind(sockfd, p->ai_addr, p->ai_addrlen) == -1) { close(sockfd); perror("listener: bind"); continue; } break; } if (p == NULL) { fprintf(stderr, "listener: failed to bind socket\n"); return 2; } freeaddrinfo(servinfo); printf("listener: waiting to recvfrom...\n"); addr_len = sizeof their_addr; while(1) { if ((numbytes = recvfrom(sockfd, buf, MAXBUFLEN-1 , 0, (struct sockaddr *)&their_addr, &addr_len)) == -1) { perror("recvfrom"); exit(1); } printf("listener: got packet from %s\n", inet_ntop(their_addr.ss_family, get_in_addr((struct sockaddr *)&their_addr), s, sizeof s)); printf("listener: packet is %d bytes long\n", numbytes); buf[numbytes] = '\0'; printf("listener: packet contains \"%s\"\n", buf); } close(sockfd); return 0; }
Your problem is this: Code: $ strace -f ./talker localhost foo [..] recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[{{len=20, type=NLMSG_DONE, flags=NLM_F_MULTI, seq=1496375815, pid=14048}, "\0\0\0\0"}, {{len=1, type=0x14 /* NLMSG_??? */, flags=NLM_F_REQUEST, seq=0, pid=0}}], iov_len=4096}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 20 close(3) = 0 socket(AF_INET, SOCK_DGRAM, IPPROTO_IP) = 3 connect(3, {sa_family=AF_INET, sin_port=htons(4950), sin_addr=inet_addr("127.0.0.1")}, 16) = 0 getsockname(3, {sa_family=AF_INET, sin_port=htons(31436), sin_addr=inet_addr("127.0.0.1")}, [28->16]) = 0 close(3) = 0 socket(AF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3 connect(3, {sa_family=AF_INET6, sin6_port=htons(4950), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 0 getsockname(3, {sa_family=AF_INET6, sin6_port=htons(51297), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [28]) = 0 close(3) = 0 socket(AF_INET6, SOCK_DGRAM, IPPROTO_UDP) = 3 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 5), ...}) = 0 write(1, "Price multicasted: 50.34\n", 25Price multicasted: 50.34) = 25 sendto(3, "50.34", 5, 0, {sa_family=AF_INET6, sin6_port=htons(4950), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 5 nanosleep({1, 0}, 0x7fff12a85070) = 0 write(1, "Price multicasted: 50.36\n", 25Price multicasted: 50.36) = 25 sendto(3, "50.36", 5, 0, {sa_family=AF_INET6, sin6_port=htons(4950), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 5 nanosleep({1, 0}, 0x7fff12a85070) = 0 write(1, "Price multicasted: 50.27\n", 25Price multicasted: 50.27) = 25 sendto(3, "50.27", 5, 0, {sa_family=AF_INET6, sin6_port=htons(4950), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 5 nanosleep({1, 0}, 0x7fff12a85070) = 0 write(1, "Price multicasted: 49.65\n", 25Price multicasted: 49.65) = 25 sendto(3, "49.65", 5, 0, {sa_family=AF_INET6, sin6_port=htons(4950), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28) = 5 nanosleep({1, 0}, ^Cstrace: Process 14048 detached <detached ...> This section of code looks for any interface to bind to and then bails after the first socket call that works. Problem is it doesn't care if the thing it finds is AF_INET6 and happily uses IPv6 to try and talk to the IPv4 listener on the other side (because you hard-specify AF_INET in listener.c): Code: // loop through all the results and make a socket for(p = servinfo; p != NULL; p = p->ai_next) { if ((sockfd = socket(p->ai_family, p->ai_socktype, p->ai_protocol)) == -1) { perror("talker: socket"); continue; } break; } Either add this in that loop: Code: if(p->ai_family != AF_INET) continue; Or get rid of that loop you probably don't even know why you're using. With that change it works "fine" for me: Code: $ ./talker localhost foo Price multicasted: 50.34 Price multicasted: 50.36 Price multicasted: 50.27 Price multicasted: 49.65 Price multicasted: 50.43 Price multicasted: 49.86 Price multicasted: 50.36 Price multicasted: 50.43 Price multicasted: 50.00 Code: $ ./listener listener: waiting to recvfrom... listener: got packet from 127.0.0.1 listener: packet is 5 bytes long listener: packet contains "50.43" listener: got packet from 127.0.0.1 listener: packet is 5 bytes long listener: packet contains "49.86" listener: got packet from 127.0.0.1 listener: packet is 5 bytes long listener: packet contains "50.36" listener: got packet from 127.0.0.1 listener: packet is 5 bytes long listener: packet contains "50.43" You're really outgunned here with what you think you're dealing with. This client/server code is some of the most basic "my first BSD sockets" program out there and this is why you're going to run into problems: Writing C++ code inside .c files and thinking it's C. It isn't C, it's some bastardized form of C with C++ mixed in. Stop doing that. I had to remove certain calls from your code just to get it to compile under gcc. Extremely basic socket handling that isn't even going to remotely scale. At the minimum an event-loop and an IO multiplexing framework is *required* (i.e. select, poll, epoll, libevent, etc). In addition everything *has* to be done in a non-blocking fashion within an asynchronous framework/paradigm. Threading at some point most likely also comes in - but NOT as a replacement for a robust client/server connection-handling framework. You're sending random things across the wire by converting floating point to a string and dumping it to the other side. This is trivial and nobody does it this way. It's not even remotely a usable or efficient protocol. You're using tcpdump to debug things when you should be using strace and ltrace w/ tcpdump as a final sanity check. Unless you've read Stevens UNPv1, and have AUPv1 and TCP/IP Illustrated Vol 1-2 on your shelf for reference you're completely shooting in the dark. You don't know what you don't know. I can only imagine what the memory handling code is going to look like. Hope you learn how to use valgrind at some point, because you're gonna need it. I have 20+ years experience in this stuff and this wouldn't even make it past a code review. You're attempting to write your own exchange but have zero real world POSIX C network programming experience. I'm very supportive of people trying to learn new things but c'mon man you need to get way more fucking realistic about what you're getting into.
So, do you want to check this into github, or should I? https://github.com/kmiklas/Ritchie/tree/master/data Imagine an exchange run by the open-source community. Linux, Gnu, Node... Ritchie! This project is a winner. Where else will you have the opportunity to build a market data server from the ground up? Also, in stark contrast to the USD6000 per month that the Nyse charges, these market data will be free, so that all market participants have access to the same level of information. Thank you for your help. Thank you.
While i960 is absolutely right if you have an IPv6 interface, there is another potential oddity I've experienced that you may run in to in testing when transmitting across your LAN (obviously wouldn't apply to a virtual network within a single physical machine) -- I've actually had some unmanaged switches that fail to transfer multicast packets. I've had a few brands and it took me an entire day of debugging code, tcpdump, etc to finally realize it was an *unmanaged* switch blocking the traffic (or rather, failing to forward it). I had this problem with a few models, but I've since gotten rid of them. Just throwing that out there in case you happen to run in to it when you start testing on different physical machines. For me an unmanaged switch that otherwise appeared to be working fine was the last thing I expected. In short, don't rule out sources of problems until you've actually spent the time validating them. As far as issues with packet encoding. You do need to define a message format. I'm going to assume you're only converting to string because you were just trying to get Beej's proof of concept code working, but do make sure to have a well defined message format that's binary. It annoys me to shit when I see ASCII based protocols where binary would be far more efficient. Yeah it might make things more difficult for developers who never wrote network code beyond using json/xml/http protocol wrapper libraries, but when writing code to custom protocols formats, binary is almost certainly the way to go. Just make sure to document it properly and think through how to extend the format while maintaining backwards compatibility or at least minimizing rewrites for those interfacing with your protocol.
I'm not an expert in networking hardware, so I'm not an ideal source. The stuff I had a problem with was what I had at home. I know one of the brands were Rosewill, so I'd not recommend them. What I currently use are Mikrotik routers and switches. A friend of mine used to work at an ISP that actually used them with good success. They're incredibly configurable and at a pretty low price point considering what you get. Another cheaper brand is Ubiquity, which has a much nicer UI. Both companies of course have lots of products available, so you buy what meets your needs. That said, when looking at routers and switches, since you'll be dealing with a large number of small packets rather than a small number of large packets, you should pay more attention to the packets per second for small packet sizes than the total switching or routing bandwidth.
Good point made re message protocol. The client/server code one can pick up anywhere, well tested, documented, and code reviewed, no need to reinvent the wheel. But the message protocol is key. Binary format a must. What worked for me is to at the very least include as initial element the size of the serialized message in order to easily deserialize and a proper message header and then the actual payload. Possibly an "end of message tag", though that is not really needed when you prepend the binary message with the message size.
Pretty much standard practice : frame, header, payload. Thanks for the link...though I feel they erroneously use the term buffer. The frame generally contains the size of the entire message, the buffer is rather terminology used for the socket to specify how many bytes are buffered and is usually way larger than a single message size.
May I inquire what you are actually trying to accomplish with this entire project? You want to share market data? Where is that market data coming from? As you describe it on your website it does not seem to be in compliance with any exchange or other data provider contract.