Scott Robertson oral history interview, 2019-02-08

VIRMANI: The following interview is conducted as part of the Georgia Institute of Technology retroTECH Software Preservation Oral History Project. Today is Friday, February 8th, 2019. The interview is taking place at the IMTC [Interactive Media Technology Center] in the TSRB [Tech Square Research Building] in Tech Square. The interviewer is Richa Virmani and the interviewee is Scott Robertson. Scott Robertson worked on the 3D Atlanta model for the 1996 Olympics. Um, so I'm gonna begin the interview with some background contextual questions to learn a bit about you. But, uh, where were you born and where did you grow up?

ROBERTSON: Um, I was born in Arkansas, um, but I grew up in Atlanta and Stone Mountain from age about three. So I've lived here ever since.

VIRMANI: Um, please tell me about your education and, or your career experiences.

ROBERTSON: Um, so I went to Georgia Tech, um, uh, as an undergraduate and 00:01:00originally in the Information and Computer Science program, um, which is no longer, and, uh, started in 1988, um, graduated in '92 and then entered the Ph.D. program in Computer Science, specializing in computer graphics. And, um, after about a year and a half in the program, um, having, I was forced to change advisors 'cause my advisor that I started out with, um, left Georgia Tech and, uh, I didn't find anyone that I really clicked with at the time and I was approaching enough credits for a master's degree and I got out with a master's degree instead of completing the Ph.D., so I graduated with a master's in computer science in '94 and then I was working, um, all through my undergraduate 00:02:00years and the two years of grad school for both GTRI [Georgia Tech Research Institute] and eventually IMTC the lab where I currently still work. And upon graduating with my master's, I took an offer to be a research scientist here at Georgia Tech at IMTC, and I've been working here for 25 years this year.

ROBERTSON: Okay. That's really cool. Um, what are your interests and or your hobbies?

ROBERTSON: Uh, outside of my work at Georgia Tech you mean? So photography is probably my biggest, um, outside interest and hobby. Um, and travel for one thing. Um, but yeah, uh, probably photography and, and travel are my two biggest interests, I would say outside, outside of work at Georgia Tech. Yeah.

ROBERTSON: And do you think, um, photography maybe had any role to play in your interest in the '96 [Olympics project]?

ROBERTSON: Oh, absolutely. Yeah, exactly. Yeah. So it's, it's informed. Um, my 00:03:00work, uh, it did inform my work on the Olympic project years and years ago and I use the, um, skills in videography and photography occasionally in my work as a research scientist here at IMTC. Um, so, and I also have an interest in visual effects, special effects. Um, I, I do that kind of on the side as well as a consultant. Um, and that has a lot of, uh, uh, a lot of aspects of photography, um, are used in the production of visual effects as well. So that's kind of, um, kind of tie-in that hobby of mine doing that kind of work as well.

VIRMANI: That's cool. Um, please describe a significant person or event that influenced who you are today.

ROBERTSON: Definitely Mike Sinclair. Um, so he was the original director of, um, IMTC when it was formed. It was actually called the Multimedia Technology Lab, 00:04:00MTL. Oh, we changed, I can't remember exactly when we changed some time in the early nineties, we changed from a lab to a center and called ourselves the Interactive Media Technology Center. Um, but yeah, Mike, um, hired me as a, uh, grad student. I worked, I think the, if I remember correctly, my junior and or senior year as an undergrad, um, under Mike's direction, although I was technically working at GTRI at the electro-optics lab at GTRI. And he, he was kind of my mentor for many years, um, especially as a, as the Director of the lab. Um, when I became a full time Research Scientist and, um, he is one of these people that he, he seems to be able to do anything and he motivates people 00:05:00to do anything. He was always saying, he never said no to any project. He always said yes, and we'll figure out how to do it, kind of thing. Um, and so that kind of motivated me to have, try to have kind of the same attitude in my career at Georgia Tech. I don't, you know, I may not exactly know how I'm going to accomplish something, especially for, um, like an industry sponsored project, but I know the team here can figure it out and we'll be successful.

VIRMANI: And when we talked to him last week, it was very clear to see that he was very passionate about all of this.

ROBERTSON: Yeah, yeah, exactly.

VIRMANI: Okay. So please describe how you initially became involved with Georgia Tech. Actually, you said that you went here for Undergrad. Yeah. Right?

ROBERTSON: Yeah. Yeah.

VIRMANI: Yeah. So were you always planning to come to Georgia Tech?

ROBERTSON: Um, gosh, that was so long ago. Um, I had in mind, um, couple different schools. I was thinking about, um, I think I applied to Duke at the 00:06:00time. Um, but when I was accepted at Georgia Tech, I had some friends that were a little older than me. Um, you know, when I was a senior in high school, that went to Georgia Tech, and it just seemed kind of like a good fit. Um, I wanted, I definitely wanted to be a computer scientist. Um, my dad was an engineer. Um, or is trained as a mechanical engineer, so he went to an engineering school and that's just, that's kind of what I wanted to do. And when I got accepted, um, it seemed to make sense to, to go to Georgia Tech and be able to stay at home and save some money. So I stayed with my family and with my parents for a couple of years before I moved out. And, um, and it was dirt cheap at the time when I started in the 1980s, especially as an in-state student. So yeah, I'm, I was, I, 00:07:00I obviously, I've probably always, um, kind of pointed at Georgia Tech. Yeah.

VIRMANI: That's cool. Um, so aside from here at the IMTC, are there any other specific programs, units, or research research initiatives at Tech that you're participating in or ever have participated in?

ROBERTSON: Lots and lots and lots. Yeah. I've had a long research career here. Um, yeah, so kind of trying to think of some that were similar in flavor to the Olympic system. So for a number of years we collaborated with the Atlanta Ballet and other dance companies to do technology-enhanced performances. Um, and we started that under Mike Sinclair's direction. Um, and that's, that's probably one of the reasons working on, um, what we called the dance technology project. Um, one of the reasons why I decided to become a research scientist at Georgia 00:08:00Tech 'cause that that project started up the year I graduated with a master's degree and, um, I wanted to work on it. So I turned down some other offers to stay at Georgia Tech, um, for lower salary in fact at the time, um, because I wanted to work on that project. And so that was doing all kinds of um, motion capture, 3D animation, visual effects and stuff on stage live, um, like instrumenting dancers with, um, motion capture technology before you could go buy it off the shelf. So we were making our own software and hardware to track the dancer's motion on the stage and then projecting effects back down onto them. Um, so that, that was all like really interesting to me and creative 'cause we were also doing the visual effects ourselves and at one of the later year performances working with artists at SCAD, um, Savannah College of Art and 00:09:00Design. So it was fun to work with them too. So the, you know, I'm not a trained artist, but I do like to do, um, artistic projects and um, and photography related and video related kind of projects. And that was one big one that spanned, I think we did that for like four or five years in a row doing performances with the Atlanta Ballet. Yeah. And um, it like hit all the, um, uh, buttons for me of doing, um, you know, interesting solving interesting computer science problems, doing a little bit of hardware design, a lot of art and um, media production, which I was interested in, you know, at one point before I decided to go to Georgia Tech when I was a high school student, I wanted to, I wanted to make movies, I wanted to go out to Hollywood and, uh, go to film 00:10:00school. But, uh, I, I didn't do that for a number of reasons, but yeah, that was kind of, those kind of projects were as close as I, um, got in my head to like making movies and doing something really creative. Yeah.

VIRMANI: So how have your interactions with such programs and projects, um, affected your, uh, personal life?

ROBERTSON: My personal life? Um, that's a tough question. Um, I mean I guess it's kept alive my love of doing artistic and creative things, um, uh, like photography and making, making my own little films and um, you know, for, for a while I had my own photography business, so I, I've, I did portraiture, um, had a, a stock photography business and stuff. And I've, I've kind of given that up since I have kids now. They take up all my time, free time, but, but that, I 00:11:00would say that's kind of affected my personal life. Yeah.

VIRMANI: And then kind of moving towards the 3D model for Atlanta, where did the concept and idea for this project originate?

ROBERTSON: Um, I am not absolutely certain, I mean part of it probably came from Mike Sinclair and the, you know, the crew of, of, uh, researchers he assembled, um, back in the early nineties, kind of a little bit before my time when I was, when I started working on the Olympics project, um, they had been, they had been funded and started for a year or two working on this concept. Um, I'm pretty sure if I remember correctly, and I came on as a, as first an undergrad students and then a little bit as a graduate student, I'm working on doing the 3D modeling and doing these landsat, um, uh, landscape flyovers for the system. Um, 00:12:00but the original idea for this virtual tour of Atlanta, I'm not sure exactly who came up with the idea. That's a good question. I don't know if Michael told you or not.

VIRMANI: I should go back and look. Um, so when did production begin for that project?

ROBERTSON: I think it was probably '88, '89.

VIRMANI: So it was like a long timeline.

ROBERTSON: Yeah. Yeah. Um, my, you know, my involvement started around 1990-1991. As a student.

VIRMANI: Okay. Um, and what was being developed in the industry at the time in terms of hardware and software?

ROBERTSON: Um, there wasn't, so there were some software programs like Hypercard 00:13:00and they're trying to remember when, I'm not sure when Macromedia Director, um, this kind of, um, multimedia scripting environment, um, came on the scene, it may have been later than that. Um, but I know the system that we developed relied on, um, this kind of hodgepodge of, of solutions including like Apple equipment, Amigas. Um, it evolved, um, control from, for instance, like controlling a, a bank of large platter laserdiscs that had a bunch of rendered 3D content and video on it, um, was being controlled by, uh, one of the things, probably an, uh, an Apple or a Mackintosh, one of the color, first color Macintosh, um, machines. And I think it was probably programmed in Hypercard. 00:14:00Um, Mike's Sinclair could tell you for sure, or Andy Quay if you've talked to him. Um, 'cause they did most of the, um, development on that software. Um, but there wasn't, if I remember correctly, there wasn't, um, this unified software and hardware environment to put together this kind of interactive experience at the time. It was combination of like writing custom software, building some custom hardware, and using the few kind of commercial, um, software packages like Hypercard and maybe Macromedia Director to like script the 2D and 3D animation.

VIRMANI: What hardware and software went into the production of this project.

ROBERTSON: So like I said, it was like using, um, apple hardware. Um, Amigas 00:15:00were used especially for, um, doing sound design, um, and some of the, uh, control of, of uh, these laserdisc players. So we had a, like I said, uh, a collection of, I don't know, probably four or five laserdisc players controlled by, um, serial connections. Yeah. And, um, there was some custom user interface devices, um, built. So we had, uh, you've probably seen pictures of the system, but we had a custom-built, rear-projected essentially a touch screen where we used a video projector, um, in this, in this pedestal. Um, shining on, you know, projecting onto a 45 degree angle mirror, which projected up onto a rear projection surface. And then we had machined this like lucite 3D model of 00:16:00downtown Atlanta. And then we built, um, as Mike Sinclair actually built it, um, used, um, little force transducers, three of them in three corners. He may have used four, but I'm pretty sure it was just three, um, in three corners of that lucite Atlanta model that could essentially measure the force of that, of that, um, plastic model and could use triangulation. So like, where are you pressed on the model, like which building you pressed or where you pressed, um, you could read the force imparted on those three or four transducers and figure out approximately where you were pressing on that model. And then we had the, the city itself, like a map, a 2D map thing, like Google Maps, um, long before Google Maps, that projected up on, um, into that frosted kind of model. So there 00:17:00was like a 3D rear projection screen and we use that to trigger events in our software. So that was all custom built, um, custom built, uh, software. And then we also had three projection screens in three projectors projecting like this very wide field, um, uh, projection. Um, so you got this almost kind of immersive cave-like experience and we had live like helicopter aerial footage shown in some cases and our 3D graphics of like we have 3D renderings of and animations of all the future Olympic venues and these fly throws over North Georgia. I was, that's primarily what I did was this landsat, um, topographic map flyovers and that was projected on these three screens, kind of synchronized 00:18:00and stitched together. And they were all sourced from laserdisc players. And so you had to keep at least, you know, for the three projection screens, three different laser players synchronized to play the three different sections of the video. Yeah. So that was all custom hardware and software built for the system.

VIRMANI: That's a really cool, um, what motivated you to get involved in this project?

ROBERTSON: Um, I just, I just thought it was fascinating and a great idea and wanted to learn like how to do 3D animation, how to, to, um, create like compelling and entertaining visuals and stuff. And it was, like I said, it was kind of close to filmmaking and, but still being a, um, you know, trained to be a computer scientist.

VIRMANI: Yeah. Um, and aside from Mike Sinclair and Andy Quay, like who are others who collaborated on this project?

ROBERTSON: Um, Ed Price, who he passed away, um, a year ago. Um, he, he was, um, uh, involved a lot on the project. We have, you probably learned from Mike. So 00:19:00we, Mike Sinclair, there were, there were a bunch of people who worked on it either who worked at Georgia Tech or as consultants. We had a, um, gentleman by the name of Frank, that I think was his name. He came from, um, the visual effects industry in Hollywood. And so he worked as a consultant on the project, um, doing things like, so we have this, um, motion capture generated animated, um, golden athlete. That's a picture over there. Um, so he was in charge of building this animated character, um, who was like, um, the intro to a lot of the, um, Olympic system video and she, you know, she was the torchbearer who ran into the stadium and stuff. And so, um, he, he was brought in to supervise like all of the 3D animation and wrote a lot of software actually for doing this 00:20:00motion capture animation. That was also, um, something that didn't really exist commercially. There was no off the shelf, um, motion capture software at the time or hardware at the time, um, that you could like a Vicon system over or something like that that you could just buy or go to a studio and, and, uh, capture this animation data. So all that had to be done.

VIRMANI: That's really impressive.

ROBERTSON: So as well. Yeah. So do you know anything about the ownership status of this project?

ROBERTSON: Ownership status?

VIRMANI: Yeah. Um, like who owns, who does it belong to or where?

ROBERTSON: Well, we just gifted. So IMTC just gifted, um, a lot of the raw film assets for the Olympic system to the Library. Um, and in a lot of the video materials, um, for it as well. So I mean, Georgia Tech owns it, you know all right. So, um, so that's kind of the extent of my knowledge, either the library 00:21:00or us, but in general, Georgia Tech owns all of this, all of these assets.

VIRMANI: Uh, what was the competitive environment like in terms of creation of the software?

ROBERTSON: Competitive environment? Um, there are, there were probably other institutions that may have been able to pull this off. I'm not sure at the time what, who they would've been maybe um, uh, like museum exhibit design companies perhaps or special effects companies might have been able to do something like this. Um, maybe, but, um, I'm feeling that the, you know, Mike Sinclair in particular was able to put together just the right team of engineers, scientists and artists to need to really pull it off that I think it would've been hard for anybody else but, uh, MTL slash IMTC and Georgia Tech to have done it at the 00:22:00time, cause it was all new, kind of cutting edge technologies.

VIRMANI: Was it ever meant for commercial use or anything?

ROBERTSON: Um, so we had, we had, um, used components, ideas, um, um, uh, um, you know, components of the system in future projects that IMTC did. Especially we had, we had a whole series of, um, more scaled down usually, but similar systems, especially for museums. Um, Michael Carlos Museum at Emory. Um, we did some, um, similar kind of interactive exhibits for the High Museum, for the Smithsonian, um, for, and we also did these, um, artistic collaborations like I 00:23:00mentioned the dance technology project where we used similar software and hardware, um, uh, and, and ideas that were kind of, um, germinated on this project in these later projects. So we never, I don't think we, I can't think of any effort to patent or, or really try to capitalize on any of any IP, um, by Licensing. But we certainly did use it internally, at least not early on the, there, there didn't seem, um, there were other that we did get patents or license IP on, but to my knowledge, not any of the work that went into the Olympic system, but I could be wrong about that. Mike Sinclair would again know the answer to that, but he's got a whole patent portfolio under his name, so.

VIRMANI: When was the software introduced to users?

ROBERTSON: Um, so the system was debuted I think, ah, '91, '92, sometime around then. Um, so it was shown in Tokyo, um, to the International Olympic Committee members. So it was part of Atlanta's like final presentation and bid to get the '96 Olympics. Yeah. So the whole system was, um, shipped over to Tokyo and, and um, debuted there. And I, I am, I want to say, I want to say '92 maybe, was when that happened

VIRMANI: And how was it received by the public and by the Olympic Committee?

ROBERTSON: There was a lot of press. There's definitely a lot of press and um, you know, I, I, I didn't go to Tokyo. Some of my colleagues at the time, I was still student, um, but a lot of the lab, um, lab, um, research scientists in my, um, made that trip. And you know, what I remember of it was the, it was really, 00:25:00really well received. Yeah. We got a lot of good press. Georgia Tech got a lot of good press out of it and it became like this traveling system for years after that, it went to France and went to all, all over the world. Um, it was hard keeping everything running cause it was all custom hardware for instance. So, yeah.

VIRMANI: And has this system inspired the creation of other software?

ROBERTSON: Oh, definitely. Yep. Yes, definitely.

VIRMANI: What are examples?

ROBERTSON: So, um, you know, like I said, we've had a series of interactive, um, kiosks and presentations and stuff. Um, like for the High Museum, we had this whole series on um, different, um, aspects of visual art that were installed. Um, at the High Museum back in the '90s. Um, we had a, um, series of interactive 00:26:00kiosks at the Emory Michael Carlos Museum, um, where they, they wanted to showcase a lot of their collection that wasn't currently on display. They had a smaller museum at the time. Yeah. And they had a lot of stuff in storage and, but they wanted the public to be able to see it, um, people who went to the museum and eventually online. And so, um, you know, a lot of the things we learned from doing the Olympic system went into those kinds of kiosks and experiences. Um, we, we built a, probably the closest replica of the Olympic system that we ever did was, um, this interactive theater experience at the philanthropy gallery over at, um, you know, that Arc de Triomphe-looking...[Millenium Gate Museum]. So over at Atlantic station, so, um, they have a, it's, I think it's been decommissioned, but, um, back in 2006, 00:27:002007, we built a similar system to the Olympic system that used multiple stitched, high def, just projection displays. And, um, we used at the time we used Wiimote controllers to have multiple people be able to like interactively explore this virtual environment that was projected on the screen. And it was all about learning about the history of Atlanta and the history of philanthropy in Atlanta and so on. And, um, so that was very similar in technology, although we're using more modern, much more modern, um, software tools and modern hardware. Um, but it was, it was definitely informed by what we did for the Olympic system.

VIRMANI: And could this system still run in its original intended form today?

ROBERTSON: I doubt it, I mean, it's, you'd have to find some vintage computing 00:28:00equipment to get it to run, um, including Amigas and you know, OS five pre, pre, um, uh, OSX based, uh, Macintoshes. Um, and finding all of the software at this point, you know, we do back things up, but, um, probably about software is on tape formats that we don't have drives to read it anymore and we save everything. But still, yeah, I mean, it'd be hard, it'd probably be easier to just, um, recreate it using modern tools than resurrecting the old hardware and software.

VIRMANI: So what future software do you imagine that this system might inspire?

ROBERTSON: Um, well, one was, you know, was some, some of them were the things that we did for museums and galleries since then. Um, you know, what we were 00:29:00trying to, where we were, we also looked at for inspiration at the time were things like IMAX films and like these, uh, these kind of, um, early, um, immersive or semi immersive theater experiences, which is what kind of what we were trying to recreate on a little smaller scale with the Olympic system. So, yeah.

VIRMANI: Um, so how did the creation of the software effect your personal growth?

ROBERTSON: Um, kept me at Georgia Tech. I mean like going from that project to these other very, um, a technical but artistic type, um, or creative projects, um, kind of kept me at Georgia Tech. I remember when I first got hired as a research scientist thinking, you know, um, I'm going to not choose the Teachers Retirement System because it has a 10-year vesting period and I can't imagine being here 10 years. Okay. Here I am 25 years later. And part of it is just because, um, we get to do kind of things we are interested in. Um, so we can 00:30:00kind of set our own research and, and, uh, um, agenda here and, um, a person like myself, I like to do a variety of things in different domains and that has kept me interested in and, uh, just being a researcher at Georgia Tech for all this time. Whereas, you know, um, friends of mine, um, people I graduated with from Georgia Tech have gone off into industry as software developers and program managers and stuff and they've gone just through, you know, they're probably on their eighth or ninth job, you know.

VIRMANI: So how would you describe the experience of working to create or discover a new system?

ROBERTSON: Um, how would I describe it? Challenging. Um, can sometimes challenging, sometimes frustrating. Um, usually ultimately rewarding. Um, I know working on these kinds of projects with, um, hard deadlines where you have a debut date or a performance date for dance technology stuff that, um, a lot of times in the days and weeks before your deadline, we would be working nights and weekends and that's tough. But it's also, um, at least when I was younger and unmarried and without family and, and um, you kind of developed this, this, um, kind of bond with your coworkers 'cause we're like sleeping on the floor in the lab, waiting for something to render and things like that. So, um, you know, I look back now with, with more like fonder memories of those times at the time I was just like a zombie with no sleep. But yeah.