Not exactly related to the interview, but I remember reading about you from some of Andy Gavin's series on Crash, and wondering what the transition from Naughty Dog to ITA was like? I just imagine that personally, I would struggle to transition from game programming where even with great engineering standards, lots of things are more 'get it done fast and close enough to right, and if something just won't work, kluge it together because we have to ship in a week and if there is a collision detection problem leading to a crash because a player jumped from this particular pixel at this particular speed, just put a crate right there so they never can jump from there' and you're doing a AAA title with less than 10 people in less than a year (or whatever the short time horizon was on Crash) to something like ITA where I would imagine there is a lot more rigor and having a tax rule incorrectly applied could be the cause of hundreds of thousands of dollars of loss for you or a client, not to mention it takes five years to build a product.
My question would be, are there any good resources you can point to that would help someone trying to understand what the truly important engineering practices a software developer needs to follow are?
You're right that it was a bit different between the two projects. Part of the reason for that, though, is that the tools themselves evolved a lot between the beginning of Crash and the middle/end of ITA. And ITA was a linux SaaS shop, so we benefited from all the improvements to tools in that stack.
With respect to resources on practices, I think most of the agile literature is reasonable. In all the domains I've worked in, the biggest challenge with doing things agile-ly has been testing: the domain complexity makes it very hard to comprehensively test the software. In the case of Crash, we just dealt with it with huge amounts of human testing (and we fixed the hundreds of reported buts). At ITA and Inky, we had/have more custom automated testing infrastructure.
One lesson is that a system like ITA's or Inky's basically can't be correct unless it's exhaustively tested.
My previous employer is in the field of network and application performance management. Even with unit-test, integration-test, and some functional-test, we can't be 100% sure we've covered everything (e.g.: different network setup) due to the domain complexity.
Having said that, "testing" has always been the biggest issue for any shop that call themselves "Agile" and it's also my screening test when interviewing with any company that claim themselves "agile": "let's talk about automation test..." :)
Testing (particularly integration testing) is just as important in non-agile shops. The difference between companies that do a lot of regular testing (whether automated or just with a big testing department) is huge.
Hi David, I was wondering if you could elaborate a little more on this thought:
"...for us to really take it to the next level and long-term defend ourselves against competition from the Sabre-s of the world, we would need to build our own reservation system and start hosting airlines. We would have to NOT be just a search provider, we would have to provide the whole platform for a major airline."
Is the sole reason for this the fact that the Sabre-s can always undercut you by providing the search for free or is there more to it? Do you know if Google will try to follow (or if they already have) that approach? Is it possible that the emergence of so many new online travel sites has changed the scope of opportunity for pure search providers?
No, that's pretty much it. I recall a specific interaction I had with a South American carrier that really colored my opinion of the future of our company. We were pitching them on our search system and it was going well for a while. But then they told us that their hosting provider had offered to lower their distribution fees through travel agents by some material percentage if they used their search instead of ours... and their CFO essentially made them take the deal.
I seriously doubt the proliferation of travel sites has changed the leverage of the parties much: you can still count on one hand the number of entities that offer all of search, distribution, and hosting. They will remain an oligopoly for some time to come, and will, I'm sure, continue to use that to their advantage.
Regarding Google (or Alphabet, or whatever they are now), I have no idea. I didn't stay on after the merger, and frankly don't ever talk to them because every conversation I've ever had with them has been totally one-sided. (They expect me to tell them stuff, and then won't say anything... because Google secrets matter!)
Travel providers could then connect directly to an airline's NDC API for scheduling, pricing and ticketing without the need for middlemen and gatekeepers.
Distribution would finally be back under the airline's control and there would be a lot more innovation.
NDC sounds awesome, and so do simplified fare rules. Are there any seriously misaligned incentives that would prevent it from happening, or is it more just general bureaucratic slowness?
The NDC spec is still evolving (and suffering from a bit of mission creep). There's also the chicken and egg problem - airlines aren't going to invest in building in-house NDC based hosting and distribution platforms until there's enough market demand for it. Market demand wont appear until airlines start distributing over NDC.
I'd have thought too that the GDS providers aren't exactly thrilled by the prospect of being easily cut out of the distribution chain... There will still be a need for aggregators though since not all travel providers will have the technical know-how or desire to develop their own NDC capable platforms/software. NDC would just level the playing field a bit.
When you were talking about the complexity of airline fares it reminded me of ATPCO. Did they exist back then or was that something the airlines later brought into being themselves?
"When we made Crash Bandicoot (with a team of 7), it was already virtually impossible to make a AAA game with 6-10 people, and that was 20 years ago.
I tell inexperienced entrepreneurs to take their honest best estimate and multiply by 10. Or, as Mark Cerny (our producer on Crash) used to tell us, "add one and increase the unit: 1 week = 2 months; 2 months = 3 years; 3 years = you're doomed".
For a less anecdotal version, read The Mythical Man Month. (The factor he arrives at is 9.)"
To my mind, yes: you use revenues to pay for new innovations and sales. It's riskier because you're making your numbers look much worse to future investors in order to grow now, and you're not putting any floor on your valuation, as you would do if you raised money now.
And, of course, when you raise money, employees can often take some money off the table. When you fund growth entirely from revenues, there isn't really a mechanism to let employees do that -- unless you're really rolling in cash and can do a stock buyback or something like that.
I wish the subject of Lisp had come up in the interview. It would go a long way toward countering the negative stereotype Lisp has in many management circles to know more about ITA being probably the greatest commercial Lisp success story in history.
