HP was my old stomping grounds. Carly saved HP, and was made the scapegoat for it. Longtime HPers absolutely HATED Carly because she insisted that all internal tech development had to make fiscal sense - that was so not the way HP had been running for 10, maybe 15 years, where tech for tech's sake ruled the roost, with an attitude of "we should know how to do everything, even if badly and unprofitably". Very little accountability. Post-Carly HP is far less techy - and far more successful - than the bloated pig HP had become when she first took the reins. The "real" HP, in fact, isn't even HP anymore, it's Agilent, and the various bits that have subsequently flown off of Agilent. And Agilent will die, it's inevitable, those old attitudes are death.
HP always understood that quality was necessary. That is the only thing I liked about the old HP... the last company I worked for was building some testers out of thirty year old HP parts they were getting from ebay.. I always used Tektronix whenever and wherever I could..
The difference between you and where I am is that I didn't inherit any analytics tools. I had to write them from scratch -and- have results in under a day. No other language other than Python can get work done in as little time as an unreasonable PHB wants. All I inherited was an execution platform that was bought with a lot of money. Second, it makes zero sense to sit around writing a parser in C++ for tick data. Even with lex and yacc, we are talking an eternity, ... or figuring out how to get boost regex working. It's still more practical to link in a python interpreter and just write 2-3 lines of code. Third, the boss doesn't program. His right hand man quant uses VBA. Writing something in VBA is completely different than C++. Presumably, his right hand man makes estimates on project completition based on VBA, which makes no sense.
Completely disagree. When it comes to making business decisions, the man cannot be a rookie or incompetent.
They weren't successful under Carly. That's because Carly didn't know how to run an R&D operation correctly. She bought COMPAQ and basically ran COMPAQ. COMPAQ, though, had all the brains from DEC which they themselves mismanaged.
if it's something you're going to do a lot, i would say it makes a lot of sense. tick data analysis is a huge part of what 'quants' do, and i spent a good chunk of time making sure the processing of it was efficient. it might not seem like a big deal on one or two issues, but when you're looking for events on baskets, computing synthetics, and running against the universe of symbols, i guarantee you, you'll start to care about that 2 sec vs 10 sec latency and multiple cores won't cure much. having said that, this gets back to what Eight was saying, there's an issue with the communication on the planning side before any work gets done. if management gives you a task without any parameters (what language to use, speed, etc.), they shouldn't really get upset when you come up with something that isn't 'what they had in mind' after the fact. you made a judgement call, right or wrong, you seem to have had the autonomy to make it. but, i think your complaint that c++ would take forever to write a parser and that python is oh so much faster, is, well... a bit hyperbolic. c++ can be as simple or as complex as you make it. using lex/yacc is an apples to oranges comparison to the few lines of python code. apples to apples would be using fstream and an istringstream object, same number of lines of code, and most likely better efficiency. python is great, but the only time i think it's ever applicable in a competitive trading firm is in one-off code or, when seeing a lot of reuse, where speed is not and will never be an issue.
The amount of time it takes to change the code to adapt for a new formatting is tedious if I have to rewrite the parser using istringstream. Log files can and do change in organizations. I'd much rather change the regex and the group ordering than go adjusting my reads out of an istringstream. Also, my beef with istringstream and primitive parsers based on things like strtok() like that is that you don't get the added benefit of having reg-ex groups. For example, you can have nested groups and then reference portions of the group. But it's not just regex groups -- dealing with repeating patterns, special character classes, etc. A 2 character change vs either coding a loop structure or changing a large static expression makes more sense. Third, if speed were really a concern, I'd just write a C++ binding to deal with the slow part -- presumably, interpreting the data once it has been parsed. Fourth, so you open an fstream. Say it's a CSV file -- C++, by default, doesn't come with classes to deal with quote-chars and such. You then iterate over the file with getline, and then you have to implement your own split() function. Is this really a productive use of your time? I don't think so. Boost helps in some regard, but now you've got a new makefile change on your hands. If you wanted to use a library, you'd have to find it, link it in, then run the program. You get a debug build, run it, and if it crashes, you need to examine the core file to find the line where it flopped. In python, you're going to simply import csv, iterate over the contents of the file, and the csv rows are returned you pre-split into an array. If it f's up, you get your line number and just go. There's no loading gdb and looking at the callstack, and wasting more time. And don't act like you don't make mistakes! Fifth, suppose you wanted to add a visualization on top. Are you going to talk to a COM interface with R? Are you going to code a GUI using a C++ GUI toolkit? Now you have to link a new library, new makefile changes. That would take forever. It makes more sense to use RPy and wxWidgets' python bindings and get the job done in half the time -- on Ubuntu, it's apt-get install whatever, then import magic. Simple as that. Sixth, NumPy and SciPy. Enough said. With C++, you have to go hunt for or standardize internally on the computational package you want to use, then adjust your makefile to link in whatever routines you want. More makefile work. Seventh, If I pay a guy $70 and hour for his quant abilities and he then spends his time writing split() or re.search()/match() over again in C++, then I'd just get angry. I would much rather invest in him learning python (or even perl) to not go through this. If he was extremely proficient with the boost regex classes, I -might- give him leeway on using C++.
And another thing: Working with time in C++ is a total waste. datetime.timedelta and datetime.datetime = simple, fast, lazy. FWIW, C# people have a good class for this stuff also. But in C++? Maybe link in another good time library? Total waste!
IMHO, unless you are engaged in HFT and have some fat Solaris machines sat ON the exchange running highly optimised C apps (i.e. are a $multi-bil hedgie) then you will get the best ROI using either C# or Java. The pool of available resources (both binary and breathing) means that these are the only 2 options that make real commercial sense to smaller players.
This I semi-don't agree with. If you're writing code to deal directly with the exchanges, you should use C/C++ and you should do it correctly. This is where there is real trading edge, because too many participants on Wall St. don't understand what makes this piece of the puzzle really work. Even if your entire infrastructure is based on some high level language, the part that is the work-horse should be engineered the most carefully in the language that gives the most bang for the buck. The beauty of most languages is that they have bindings that make it easy to create interfaces to something written in C or C++, or you can just open a pipe/socket and use your carefully crafted engine. OTOH, you may be right, as hiring people who know how to do this is very hard. The best people for this are in Silicon Valley, not Wall St.