Settings

Theme

‘The Billion Dollar Code’: Developers Who Sued Over Google Earth Algorithm

decider.com

152 points by bwindsor 4 years ago · 83 comments

Reader

remh 4 years ago

Started watching the show, it is way too dramatized but still interesting.

The actual demo of Terravision is indeed mind blowing if you remember how powerful computers in 1994 were: http://www.joachimsauter.com/img/big693.mp4

  • ehsankia 4 years ago

    > Started watching the show, it is way too dramatized but still interesting.

    I don't mind dramatization, but 4.5h seems quite steep for such a simple video that seems like it could be covered in a 20m Youtube video. Do you have a rough review if you ended up watching more of it if it's just really dragged on or actually contains a lot of content?

    • brabel 4 years ago

      I actually found it quite appropriate, specially after it picked up pace from the end of the first episode... didn't feel at all like it was dragging on.

    • smudgy 4 years ago

      If you just want the tech stuff, it's very sparse but I greatly enjoyed the connection between the two protagonists.

      It was also a nice way of teaching my wife about what the world of computers was like in the 90's.

  • the-rc 4 years ago

    That didn't run on a PC, it ran on Onyx servers, which had over 10 graphic processors.

    • cik 4 years ago

      We ended up taking our Onyxes, years later and turning them into fish tanks. I miss those days.

    • piceas 4 years ago

      Flight sim 5.0 ran on the PCs of the day and wasn't that far behind.

  • LordOfWolves 4 years ago

    This is quite impressive for ‘94, thank you for sharing.

    I found it funny - in a “how ridiculous/silly” kind of way - that at 4:40 we get to look inside a building by entering through its windows, but here we are in the 21st century with that exact capability. Individuals providing photos and/or footage of the interior of their commercial space does not seem so silly these days.

  • ur-whale 4 years ago

    > mind blowing if you remember how powerful computers in 1994

    In 1994, SGI workstations could pull off this kind of stuff rather effortlessly, the only "tricky" part was to get the LOD algorithm right.

    So not that mind-blowing in practice.

    It would only amazing if Terravision ran on a stock PC (did it?).

    • the-rc 4 years ago

      Jones and Tanner created clip mapping, first in SGI hardware, then in software. That's what made Keyhole and thus Google Earth possible on consumer machines. Is the allegation that it was all stolen? They could have invalidated SGI's patents. And no, Terravision didn't run on a stock PC, it ran on Onyx hardware. Plus it relied on an ATM connection! Even then, the demo has a bunch of glitches and gaps, plus the zooming is so slow, all of which doesn't look like superior technology for Intrinsic/Keyhole to copy.

  • Genego 4 years ago

    I tried hard to enjoy it. I did like the characters, but the dramatized story really didn't work for me. I guess it was a win though, because previously I had never heard about Terravision.

  • jeffrallen 4 years ago

    Also, check out the bow tie on the guy doing the demo. Snappy.

w0mbat 4 years ago

There were a lot of 3D globe products being created in the 90s.

I was one of the creators of Electronic Arts 3D Atlas which shipped in spring 1994. It sold two million copies on Mac and won many awards.

EA was the publisher but we developed it in London working at a BBC spinoff called MMC and getting the planetary data from UCL. This UCL unit eventually spawned Planetary Visions, a startup we worked with to supply data.

That wasn't even our first 3D zooming spinning satellite globe product. We did a huge touchscreen installation for the Kunst und Ausstellungshalle museum in Bonn, Germany. It was called "Erd Sicht" (Earth View), and in the early 90s had amazing satellite Earth graphics and animations.

  • w0mbat 4 years ago

    Other products I can think of in that early 90s period included "Small Blue Planet" from Now What Software and "Virtual Museum", a tech demo CD-ROM from Apple that included a real-time rendered globe.

  • atoav 4 years ago

    Yes, but they they did all use different indexing schemes than the patented one for Terravision.

    • avibarzeev 4 years ago

      ART+COM's patent was invalidated in 2017 due to prior art from another app called TerraVision from SRI shown in 1994.

      • atoav 4 years ago

        If you trust the guys involved (and I do because they have a great track record within the German hacking community) the story goes as follows: This invalidation was carried out by a jury who had no clue what they were doing, after the first judge (coincidentally married to a google lobbyist) pulled back frommthe case which was running rather well for them.

        I just spent 3 hours on listening to a (German) podcast on the matter with one of the people who developed the algorithm that was copied (addressing scheme for storing and adressing the different LOD tiles in memory, coincidentally the exact scheme Google uses till today, because the guys spent a lot of time optimizing).

        The prior art TerraVision had the same name but was a completely different software (military simulation if I understood correctly) and didn't make use of that algorithm.

        They still convinced the jury : )

native_samples 4 years ago

I worked on Google Earth for a while, some years ago. It doesn't really make sense to talk about it in the way the article (and presumably movie) does. There are lots of "algorithms" in Google Earth which are critical to its operation.

Probably the biggest unsung algorithmic hero of GE is the image processing and serving pipeline. The actual client tech itself was largely worked out by the time Keyhole was bought (hence the high age of this lawsuit), and the biggest upgrades to the client came mostly in the form of optimizations and support for new features like 3D buildings. What wasn't visible was the enormous amount of work in the infrastructure required to actually process and serve the whole globe's worth of imagery. At the time, Keyhole's processing and serving pipeline just didn't scale to the amount of imagery actually available for purchase. That's no knock on their engineering skills: imagery is expensive and given their business model there simply wasn't a need to process all the imagery that existed. But once Google bought them, they were given a blank cheque and told to quite simply acquire it all. The infrastructure of the time (MapReduce, BigTable etc) were a perfect fit for this problem and allowed a rapid drain of the imagery backlog that had by then accumulated. Over time more and more of the calculations done to the raw sensor data (like ortho-rectification) were pulled in house for cost reasons and to allow further scaling - the satellite imagery providers could easily become CPU bottlenecked themselves.

This sort of article/movie seems ultimately to have some sort of ideological driver, given that its opening thesis is "for every winner, there are lots of losers; the people who blazed the trail for the others to ruthlessly make billions from their efforts". I mean, that's not true is it? This blasé statement implies the tech industry is negative-sum but it isn't even zero sum, it's positive sum: the whole reason it's successful is that it's a massive creator of wealth. The assumption that the big winners in the tech industry got there simply by aggressively stealing everything is so far from reality it surely makes the article closer to propaganda than anything else.

ploek 4 years ago

If you understand German and like 4 hour long podcasts, Tim Pritlove just released a CRE episode about Terravision where he speaks with Pavel Mayer: https://cre.fm/cre222-terravision

(I haven't seen The Billion Dollar Code or listened to the podcast yet, but apparently the character Juri is (loosely?) based on Pavel: https://twitter.com/pavel23/status/1447622978859061252 )

amayne 4 years ago

Here is SRI’s project from 1994 that showed similar capabilities (and the same name) and was one of the reasons the court decided against the claimants in their lawsuit against Google:

https://www.sri.com/wp-content/uploads/pdf/778.pdf

  • BrianMcCl 4 years ago

    I wonder why the footer said it is dated to January 26, 1995 (after ITU in Kyoto 19 September, 1994) while the first page said April 22, 1994?

    This PDF file is encoded PDF format version 1.3 witch was released in 2000. It is coherent with the informations stored in the PDF heather fields telling us this file was created 02.02.2000, 10:20 and last changed 02.02.2000, 18:20 by Acrobat Distiller 4.0 .

    So at lest we can say this PDF file here is not an authentic one from 1995, witch of cause not necessarily means there could not be another PDF file somewhere else(?).

    • BrianMcCl 4 years ago

      Well I found in internet archive an older version of the same document in PS format that was captured at May 16, 1997 : https://web.archive.org/web/19970516152226/http://www.ai.sri...

      This PS still contains same dates. On footer dates January 26, 1995 while the first page says April 22, 1994.

      I also found the old SRI TerraVision homepage captured at May 16, 1997: https://web.archive.org/web/19970516142933/http://www.ai.sri...

      As well as the old datacenter page referring to SRI Terravision captured at June 5, 1997: https://web.archive.org/web/19970605180747/http://www-itg.lb...

      And here is the newer homepage of the later program version SRI TerraVision II captured at December 02, 2000 still containing some short promo videos of the software: https://web.archive.org/web/20001202100300/http://www.tvgeo....

      I need to check this site more deep but on first impression the SRI TerraVision I in 1997 and SRI TerraVision II in 2000 as shown in the linked videos are kind of inferior in sense of performance and capability compared to Art+Com TerraVision in 1994. Especially the videos from 1997(running on SGI) looked really bad and more like an early prototype. Just the 2000(on PC) videos SGI TV looked a bit better. All the TerraVisions programs may share the same name for whatever reason(?), but I would assume they do not use the same Algorithms. If Keyhole/Google "borrowed" ideas for Google Earth I would guess they did not copy them from SRI TerraVision, but from Art+Com TerraVision. But how knows?

      If Brian from SGI was the connection to Keyhole/Google he could be also the connection to the MAGIC project from SRI since they use the same SGI Onyx at the beginning as Art+Com.

  • unixhero 4 years ago

    Who were SRI?

    Why did it have the same name as the original Terra Vision?

    • BrianMcCl 4 years ago

      Stanford Research Institute https://en.wikipedia.org/wiki/SRI_International

      The same name is a really good question. But since SRI TerraVision and the Art+Com TerraVision both seems to be started at the same time around 1993 and both use the same SGI hardware (an Onyx) I could image that Brian from SGI was in contact with both teams.

      • amayne 4 years ago

        This SRI document is based on research funded by a ARPA grant:

        "This work was supported in part by the Advanced Research Projects Agency under Contract F19628-92-C-0071."

        Which implies that it was in development long before the date on the PDF.

        Earth mapping was a very hot field in the 1980s with most of it being funded by intelligence agencies and classified.

WelcomeShorty 4 years ago

Tried to watch it, did not like it at all.

Poor dialogs, poor acting, lots of technical bla bla makes hardly any sense.

JamesUtah07 4 years ago

After reading the article it wasn’t clear to me. Is this Based on a real story or is it entirely fictitious.

  • asdfasgasdgasdg 4 years ago

    Art+com v google did happen and is findable via a web search. What's not clear to me is whether this is anything more than a typical NPE/patent troll lawsuit. (Google won the lawsuit and the patent claims were invalidated because of prior art or other reasons.)

    • ascar 4 years ago

      As I get it from reading around Wikipedia they actually had something very similar to Google Earth before Google Earth/Maps happened. So I don't think it's fair to bring this in connection with patent trolls.

      However, Google didn't seem to copy any of their tech but just implemented the same idea (a searchable/zoomable globe/map), which doesn't strike me as surprising as the idea isn't too farfetched and the tech is very much related to 3D graphics and game development, which had already come pretty far by 2004.

      So the trailer implying they actually had it first and Google couldn't have done it without them seems pretty peculiar to me.

      • jmgao 4 years ago

        Looking at the federal court decision [1], it looks like the patent was predated by an implementation that was demonstrated at SIGGRAPH, and apparently the creator of that gave the source code to the plaintiffs?!

        "Lau further testified that, at the SIGGRAPH '95 conference, he performed live demonstrations of SRI TerraVision to at least 500 people, and in fact “gave [ ] the source code to TerraVision” to Art+Com employees who were in attendance and “walk[ed] them through the source code.”"

        1: https://caselaw.findlaw.com/us-federal-circuit/1878050.html

      • asdfasgasdgasdg 4 years ago

        AFAICT it was only a tech demo. They describe it as a "research project" commissioned by Deutsch Telecom on their website. I don't see any indication that this company were actively marketing or otherwise deriving economic value from it when they did the lawsuit, or even when Google Earth was first announced.

  • sheinsheish 4 years ago

    Displayed on the end of episode #1:

    “Terravision was a joint project from Weathernews International (Hiro Ishibashi, Andreas Schneider), ART+COM and DeTeBerkom. The series is a fictional adaptation of the events at that time and arose from the court transcripts as well as conversations with Joachim Suater, Pavel Mayer, Axel Schmidt, Gerd Grüneis, Martin Sibernagl.”

  • marianov 4 years ago

    Yes. And it was discussed on HN: https://news.ycombinator.com/item?id=7301630

  • avibarzeev 4 years ago

    It's a highly one-sided version from people who rightly IMO lost their case and had their patent invalidated.

    The show left out Keyhole, which actually wrote Google Earth before Google bought them. It was written 100% from scratch and based on different and better technology.

    https://avibarzeev.medium.com/was-google-earth-stolen-7d1b82...

  • carbonx 4 years ago

    Wikipedia says based on true events.

    https://en.wikipedia.org/wiki/The_Billion_Dollar_Code

ryanlol 4 years ago

I largely skimmed through the series, but the way this portrays the german hacker scene while simultaneously appearing to advocate for software patents feels very strange.

The series portrays these guys as the heroes, but in reality they just seem to be patent trolls who fortunately lost.

A lot of really cool retro tech stuff to spot though, they certainly weren’t lazy with this even if the politics of it seem weird.

  • tagmaduut 4 years ago

    "patent trolls who fortunately lost"

    It's of course okay to express this opinion. But given all the facts on the table dismissing them as mere patent trolls seems quite unfair. I encourage you to read up a bit on these people, who are/were i. a. accomplished art professors and programmers.

  • SilicaAndPina 4 years ago

    Isn't that how all the people from computer clubs in the 90's turned out?

    Like apple came from that shit, heck the entire state of hacking right now with corporation simping don't even get me started on "bug bounties" and "Responsible" disclosure

jpomykala 4 years ago

As a programmer, I must say I really enjoyed the movie.

hello4353 4 years ago

How did they get were able to acquire and digitize so many maps in just 4 months according to the show. That part was stange. They say they hacked it from Nasa site, but i think its made up, could not find the source.

marianov 4 years ago

I found the references to 90's tech fun (like Chaos Computer Club).

unixhero 4 years ago

At the show in Kyoto... Does anyone know what kind of screenshot they are using for their demonstration? They have three of them standing next to each othrr. They are pretty magnificent.

lalaandlife 4 years ago

Entertaining and seriously demonstrates to us all how tiny we really are on this planet and without push, shove and money and judges in cahoots with others against you what chance does anyone have in court? That’s why I became an arbitrator and make judgements on natural justice as opposed to the law.

pegyong 4 years ago

As I have finished watching the show, it's like it is hyped up at a certain point. It's quite interesting, but as a developer, some terms don't make sense.

BarbEh 4 years ago

Would this case have been won or lost if Terra Vision had patented in another G-7 country and the case had been heard under the law of that country?

a-dub 4 years ago

i'd love to read this story, but with all the scrolljacking, pop-ins, video ads and other ridiculous distractions it's basically impossible to do so on my phone.

good job.

varelse 4 years ago

So Google Earth bought a company called keyhole that was formed by the same people that had done the Mars demo (if you saw this in the '90s it was amazing and you could tell the demo alone had a future even if the company crashed) for SGI which then became the basis of that startup which then got acquired by Google. And all of that launched the political career of Brian McClendon, a former SGI manager. Another of its founders went on to be part of Roblox.

  • sanguy 4 years ago

    Keyhole also had some money in them from In-Q-tel which was also a heavy Google investor at the time. They wanted the concept of a searchable globe for intelligence purposes.

    Also the founder of Keyhole, John Hanke, came from the MUD world and now responsible for Niantic and the Pokemon game. A true visionary of blending the real and virtual worlds.

  • DanielleMolloy 4 years ago

    According to the lawsuit, McClendon and Michael Jones had access to the code of Terravision: https://artcom.de/news/artcom-sues-google/

  • marianov 4 years ago

    Google Earth traces back to the Mars demo? That was amazing when I was a kid

rob74 4 years ago

Er... it's not entirely clear, but apparently the infringement claim was based on "the ability to zoom from the globe graphic all the way down to a particular neighborhood"?! So something a mildly talented programmer could come up with in (generously estimated) 1-2 days tops? And sure it looks cool, but claiming this animation is original enough to be patentable is stretching it a bit...

  • seabird 4 years ago

    I strongly encourage you to try and achieve this in less than 24 working hours with the hardware and software that was available at that time. Even if you "cheat" and use high-end workstations and ~25 years of new knowledge and hindsight, even a talented programmer would likely struggle with this problem for multiple months.

    • ffhhj 4 years ago

      Hmm, tech from '95... using VESA BIOS extension and high/extended memory modes, you might be able to modify Doom source code for the optimized 3D operations, but you'd still need to create a LODed-megatexture database and an API to serve it, unless you want users to "Insert CD 23 and press Enter".

  • arthurcolle 4 years ago

    Is this what you imagine?

    ```

    import gis

    gis.renderGlobe()

    ```

    the actual tiling logic, data storage requirements, performant rendering in something like a shitty 1990s browser... I don't think you have an appreciation for the difficulty of bootstrapping this ex nihilo.

    • rob74 4 years ago

      Well, the article (more or less) says that it ran on an SGI workstation (some photos are available here: https://artcom.de/en/?project=terravision), of course it would have been impossible in a 1990s browser. What I actually wanted to say was that, while the technical feat of programming something like this in the nineties is certainly something to be proud of, I still think that the basic idea of zooming in from orbit to a certain section of the globe is too obvious to be patentable - and from a purely mathematical point of view (ignoring things like levels of detail of 3D models, memory limitations, the actual 3D programming etc.) the signature zooming-in animation from Google Earth (which apparently was also present in Terravision) is not that complicated to implement either.

  • jandrewrogers 4 years ago

    This was definitely non-trivial at the time. Even now the only way it would take "1-2 days tops" is if you use a naive design that ignores important edge cases and leverages a lot of code and computer science that did not exist then. Geospatial is hard.

  • bpodgursky 4 years ago

    I don't disagree that it's not patentable, but high-performance GIS / tilemap scaling manipulation is not nearly as simple to build as you are imagining.

  • perl4ever 4 years ago

    "The problem, once solved, will be simple"

    (possibly Thomas Edison, who I believed was accused of stealing IP a lot)

  • ahazred8ta 4 years ago

    There's (analog) prior art for the zooming from 1957, 1968, and 1977. https://en.wikipedia.org/wiki/Powers_of_Ten_(film)

  • jebronie 4 years ago

    So easy! Why didn't they just import it into Unity???!!! Thats you.

  • wrnr 4 years ago

    As generous as the half a bowl of rice Kim Jong-un gives to his loyal subjects.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection