Web Masters Episode #77: Steve Madere


Google Groups - Wikiwand

Steve Madere:

It could have been Google several years earlier … In terms of like, I could have made a better search engine for the internet that’s faster than everything else that existed at the time, and I say, “for the internet.” The thing is, I was tackling the harder problem because Usenet was significantly larger than the web. So I thought, well, everybody and their dog is going to be able to index the web. That’s puny. Only my tech can index Usenet so I’ll have this market to myself. But as it turns out, it wasn’t the greatest market to have to yourself.

Aaron Dinin:

Yep. In the early days of the internet, the worldwide web just wasn’t that big of a thing. So even though we’ve spoken with lots of people here on Web Masters who built companies by indexing and searching the web, from a technical perspective, indexing and searching the web wasn’t a particularly complex problem to solve. There was actually a bigger digital data set to search. It was called Usenet. And on this episode of Web Masters, we’re going to hear from the person who built the definitive search engine for that. His name is Steve Madere and he founded a company called Deja. It could have maybe even become Google until, well, a little thing you might have heard of called the dot-com crash. Google ultimately bought Deja and turned it into Google Groups. That’s the short version of the story anyway. I think you’re going to enjoy the long version. Are you ready to hear it? All right. Let’s get dialed in.

[INTRO]

Aaron Dinin:

Welcome to Web Masters. I’m Aaron Dinin, your host. I’m a serial entrepreneur and I study the history of tech entrepreneurship at Duke University. This podcast comes out of that work. It’s where I share it with all of you: Stories of how the internet’s most impactful innovators built their companies. We all get to listen, learn, and as a result, hopefully become better entrepreneurs.

On this episode, we’re talking with the founder of one of what you might call a “could have been company,” as in it could have been enormous had a few small twists of fate gone a different way. These kinds of stories are, in my mind, at least, some of the best to learn from because they offer great insights into the kinds of decision making any great founder is going to have to successfully navigate in order to build something truly world changing.

But before that, while I’m on the subject of building successful companies, I want to take a moment to tell you about our sponsor because well, they help people buy and sell successful companies. Web Masters is being brought to you with help and support from our sponsor, Latona’s. Latona’s is a boutique mergers and acquisitions broker that helps people buy and sell cashflow positive internet businesses and digital assets. That includes everything from e-commerce stores to content websites, Amazon FBAs, Etsy shops, SaaS apps, domain portfolios, eBay storefronts. Basically, if it’s a profitable online work from anywhere internet business, Latona’s is going to be able to help you sell it and get it sold for a great price. Or if you’re hoping to buy an internet business, Latona’s can help you with that too. Just check out their website. You’re going to see listings for tons of profitable internet businesses they’re currently helping to sell. You can find one that’s going to be perfect for you. Just head on over to: latonas.com. That’s L-A-T-O-N-A-S.com.

In a way, this episode of Web Masters builds off of a previous episode we pulled together about a year ago and helps paint the picture of what would’ve been essentially a completely different version of the web. The episode I’m referring to is Episode Number 29, featuring the story of Tom Truscott and Usenet. Usenet was like a different version of what the web became and that version of the web would have needed a search engine too. Kind of like the worldwide web needs Google. This episode is about the entrepreneur who built that search engine for Usenet, Steve Madere, and the company he built was called Deja. So in order to understand Deja, let’s start by learning a bit more about Usenet.

Steve Madere:

Usenet was a system for people to sling around giant email distributions that were too big to actually fit everybody’s email addresses in the two line. You could send a message to Usenet and it would go to thousands of computers worldwide, where anybody running a Usenet news reader could read the messages. So it was a lot like a really big mailing list. People would connect computers to each other with modems, and it would relay a huge pile of messages on a gazillion different topics amongst all these computers, and the whole set of computers connected together transferring around all these foreign messages was called the Usenet.

Aaron Dinin:

So yeah, Usenet was like a huge collection of mailing lists about basically any topic you can imagine. Everything from coding to cooking. To borrow a description from another Web Masters guest, Justin Hall in Episode Number 31 … Justin, by the way, is the man widely considered to be the first successful blogger. Usenet was basically like Reddit, but with a lot fewer pictures. Usenet was incredibly popular, at least with technologists, back in the 1980s and early ’90s, which was before the web existed. In fact, many of the world’s most prominent and impactful technologists first discovered their love of online networks thanks to Usenet. That includes lots of people we’ve spoken with here on Web Masters as well as people like Linus Torvalds, who announced Linux to the world on Usenet and even perhaps, somewhat ironically, Tim Berners-Lee who first announced the worldwide web, the thing that would pretty much kill Usenet via, well, Usenet. One of the other technologists on Usenet who was there pretty early on was, of course, this episode’s guest, Steve Madere.

Steve Madere:

I got into networking in the early ’80s. In the very earliest days, we were connecting a bunch of computers together via something called an RS-232 port, which is a joke. Then ethernet was like this amazing thing. “Oh wow, you can have 20 computers connect together.” I actually worked at a small consulting firm that basically helped small businesses configure their networks and that was where I discovered Usenet. I basically became addicted to getting the answers to all of my technical questions from all the best technologists in the world by asking them on Usenet.

Aaron Dinin:

About how early were you using Usenet in relation to its general historical growth arc?

Steve Madere:

Well, I first started using it back in the modem to modem days in the 1980s and that was even before most Usenet nodes had access to the, what was then called ARPANET. The “internet” really started closer to 1990, maybe even after that, which was basically, ARPANET getting opened up to everybody. It was approximately at that point that Usenet made the transition to being mostly communicated around by nodes on the internet, instead of being communicated directly from one host to the other via modems.

Aaron Dinin:

And what made you so interested in Usenet?

Steve Madere:

Well, the great thing about Usenet was all the brightest minds of software development were on it and they were trading tips and techniques and advice constantly. If you go and look back at the old archives of comp.lang.c, there’s a bunch of very famous people in there, trading advice on how to do things in software development, the same thing during the initial development years of C++, et cetera. So it was extremely useful in actually getting real things done … effectively, is what it came down to. In that way, it was very different than the forums you’d find on AOL, et cetera, where people are just chatting with each other and it’s one third illegal activity and one third deplorable activity. People who actually had something to do trading advice on how to get it done.

Aaron Dinin:

Really? That’s not what I’ve heard. I’ve heard there was plenty of let’s call it less than intellectual conversation on Usenet.

Steve Madere:

Yeah, that’s true. Okay. All the “alt groups,” alt this, alt that, some of them were hilarious. Others were disturbing. True enough. They were various TV fan groups like alt.wesley.crusher.die.die.die, Which was a fan fiction group created entirely for the purpose of writing fan fiction about terrible ways for the Wesley Crusher character on Star Trek to die.

Aaron Dinin:

Okay. So in order to fully understand this episode and the significance of what our guest created, it’s important to understand how Usenet was originally built differently than the web. The web consists of information that stored remotely on a bunch of search servers and all of us as users access those service via our browsers. Usenet was originally what’s known as a store in forward system, which is a bit different. Basically, it means there were a bunch of nodes that contained full copies of everything on Usenet. When someone would post to a news group, at Usenet’s term for categories, that post would be forwarded along with any other new post to the rest of the network over the course of the next few hours, or in some cases, days, while the computer doing the forwarding would also retain an entire copy itself. At least that’s how it originally worked.

Steve Madere:

When Usenet first started, it was low enough volume that everybody kept everything. And then after the volume going through it got to be high enough, people started lopping off the older part of history. And the window kept getting narrower and narrower until Deja came out and we reopened it back up to everything since the beginning,

Aaron Dinin:

As you heard, the more popular Usenet got, the harder it was to retain all the info, which was part of the problem Steve was trying to solve when he launched Deja. The other part of the problem was that at this point in internet history, there wasn’t really a concept of archival information. In other words, that sense you could always go back and search through what had been posted on the web years and years ago, well, that wasn’t really there on Usenet.

Steve Madere:

The whole user workflow in Usenet was basically to watch a stream of messages on a specific topic going by, and you had to be there to see the answer. And people are so used to everything being archived now; they have no concept of what it was like to have stuff just go by and you never see it again. But that was the way things were happening on Usenet. Nobody had enough storage to keep around the entire history of Usenet. It was pretty expensive to store data at the time. It was a very large volume of Usenet traffic going through. And so your typical user’s experience was messages would arrive, they would hang around for two weeks to a month and then disappear. And if you didn’t happen to be reading the news group where the answer was that you wanted during that period, you just never knew it went by.

And even if you did see it, if you didn’t have it memorized, it was very hard to go back and find any given message, right; for one thing, if you didn’t personally archive a copy of a message yourself, it was going to disappear in two weeks to a month. Deja was actually the first publicly accessible utility that had any kind of history of what was in Usenet. Plus we made it searchable. We collected archives from a bunch of system administrators from well-funded sites that had actually had the resources to put away copies of what was going by on Usenet before deleting it. And they sent us archives of their stuff and we just kept adding it to our index.

Aaron Dinin:

So that’s actually pretty wild to me. I guess it hadn’t really occurred to me that there was a time in internet history when all the content online wasn’t just thought of as some sort of everlasting repository of knowledge and information.

Steve Madere:

Exactly. Before we started out Deja, nobody had more than a couple of weeks to a month online. A few people had some archives on tape that they pulled aside. We spent about a year accumulating the traffic as it went by and adding it to our search engine. By that time, it was already vastly more than anybody had ever seen collected in one place before. And you know, if you’re going to have that archive of information, it’s got to be equally searchable, right? So simultaneously coming out with a search engine and with the reason to archive all that stuff and make it publicly available like that were the two parts that were both necessary to make the other one worthwhile.

Aaron Dinin:

And then I assume it was Usenet’s lack of persistence that pushed you to creating Deja. Is that right?

Steve Madere:

Okay. So I knew Usenet was incredibly useful to me and I thought, how much more useful will it be to me when I don’t have to actually see every discussion in comp.lang.c. I can go back and actually find the previous discussions of the same issue I’m looking for. And then I don’t have to ask everybody my question and sit and wait for the expert to come online and see my question and answer my question. I just see when he answered that same question from somebody two years ago. That was my motivation in creating it. If it’s that useful to me, it’s got to be that useful to a lot of people. But mostly, I expected it to really useful to technologists. I thought that all the other people who were trading binaries and talking about Wesley Crusher, they didn’t really care about searching through the history.

I thought this was going to be mostly useful to technologists. So Deja started with a technology. I was doing text retrieval research in my day job and came up with an idea for a highly scalable search engine. As it turned out, it was pretty much the same architecture that Google search engine eventually came out with. But I built it, and then started testing it on various data sets. And the only dataset I could find that was big enough that anybody would care about was the entire history of Usenet. At that time, the entire history of Usenet was several times the size of everything on the worldwide web. This was in 1995.

Aaron Dinin:

Oh, okay. So it was kind of like a practical experiment? Usenet was a huge open data set. And I suppose that made it a good resource for testing your technology. Now, am I understanding that correctly?

Steve Madere:

More specifically, I had been working, when I said I was working in tech retrieval research, I was working on a practical business problem of what they call, “support problem rediscovery.” And IBM’s AIX development group, they had an enormous number of support tickets coming in and customers would call in with a problem and they had hundreds and hundreds of support people on the phone. And in order to figure out whether a problem had occurred before and they already knew the answer, they needed a search engine to search through all of the previous trouble tickets. So I was working in a research group working on something called semantic similarity search. And so that was in about 1992.

Semantic similarity search really had a big resurgence in actual commercial use in about 2004. So it was about 12 years ahead of where the rest of the industry started actually making use of it. But, at any rate, the main thing was I had figured out how to make a highly scalable search engine that could do keyword searches very efficiently, but also could do phrase searches very efficiently, which was unusual at the time.

And in fact, I believe, I know I’ve looked it up before, but I believe you can go back in Google Groups and search the history of the news group com.lang.tcl and find my announcement of the original release of Dejanews in which I was basically asking all the people who were TCL developers, hey, can you guys go and beat on this thing and let me know if it works for you?

Aaron Dinin:

And how did they respond? How did you get your first users for deja?

Steve Madere:

Okay. So very early on, I identified that far and away, the easiest way to get word out about Usenet search engine is on Usenet. And I also knew that people on Usenet are generally very protective of it. Several people had tried to use it. This is where the word spam came from … was from posting stuff to Usenet that was off-topic that was basically advertising. So here I knew that Usenet was the best place to tell people about a Usenet search engine. But had to be very careful not to appear to be self-promoting and advertising on Usenet.

So my solution was to hire somebody to search Usenet for new questions that just arrived. Then search Usenet for the answer to that question. Then post a post that says, hey, I found the answer to your question here. And there’s a link to Dejanews with the search. I did a search for this and found this. And I told him your attitude in every one of these posts needs to be like, oh yes, this is just the way it is. Everybody knows that’s how you find stuff on Usenet. Like, let me show you how to do this, newbie.

Dejanews had existed for a few months and it had been growing somewhat, completely organically, from the people I first introduced to it at comp.lang.tcl, then passing it around to other people. And then I hired this guy and from just maybe two months after the availability of Dejanews, he’s posting as if everybody should know this because it’s been there all their lives. We did manage to grow very exponentially that way. Because very soon thereafter, other people started mass pointing to people about how they could use Usenet.

Aaron Dinin:

Wow. That’s one of the first examples of what we might call a successful growth hack. That’s pretty cool. All right. So you started getting all these users, but this was a business, right? So how were you thinking about making money?

Steve Madere:

Banging my head against the wall about how do I monetize this? I knew that technologists are notoriously cheap about paying for things, especially if they could imagine they could build it themselves. Like, hey, I could spend three years building that and have it myself. I’m not spending two bucks for that. So I beat my head against the wall for a while and finally came up with, wait a minute. There are these technology magazines out there, Byte and PC Magazine and things like that. And they completely live off of advertising revenue because they’re reaching IT people.

All these IT people would totally use this search engine. Plus, if they use this search engine, you’d know exactly what they were interested in based on the keywords they’re using and the news groups they’re looking at. So we could make the world’s best possible advertising. And so this was in 1995 that I put this together as a business plan. And I wrote it up and took it to some local VCs. And apparently, that was approximately the same time that Yahoo was coming up with pretty much the same business plan. And so that’s how we managed to get funding. We and Yahoo, and two other companies were kind of at the vanguard of … we can turn eyeballs on the internet into advertising dollars.

Aaron Dinin:

So basically it sounds like you had the exact right idea except by focusing your technology on Usenet rather than the web, which was what Yahoo and AltaVista and Lycos and all the other search engine e-portal companies were doing, you basically bet on the wrong winner. Why didn’t you index the web? Why did you choose Usenet?

Steve Madere:

Yeah, that was me being stupid. Really, I could have been Google several years earlier in terms of like, I could have made a better search engine for the internet that’s faster than everything else that existed at the time. And I say for the internet. The thing is I was tackling the harder problem because Usenet was significantly larger than the web. So I thought, well, everybody and their dog is going to be able to index the web. That’s puny. Only my tech can index Usenet. So I’ll have this market to myself. But as it turns out, it wasn’t the greatest market to have to yourself. Because a few years later, people started creating web forums.

And interestingly, the reason why web forums took over from Usenet was because there were all these companies who each wanted to own a web forum. And you couldn’t own a chunk of Usenet, but you could own a web forum. And Google’s habit of indexing every forum post on every web forum, basically, took over the market space of what Dejanews had covered. So stack overflow today completely takes the place of comp.lang.* on Usenet, so.

Aaron Dinin:

In a way, it sounds like the web ultimately supplanted Usenet because of basically capitalism. You had all these individual companies wanting ownership of their little relevant slices of network space. And the technology of the worldwide web was much more conducive to that as opposed to Usenet, which was nonownable. Is that what things were like?

Steve Madere:

Yes, yes. That is. Usenet had a critical mass of participants in a whole lot of different topics. And if it weren’t for the fact that there were a thousand highly motivated, highly funded companies, each trying to steal a chunk of it, it probably would’ve maintained that. But it got ripped apart by one forum at a time, basically, being taken over by some company that invested enormous amount of money and improving the quality of it, with moderation and adding graphical content and stuff like that and making it really easy to view it on the web.

Dejanews actually did make Usenet really easy to use on the web compared to how it had been to use Usenet, having to get a special Usenet client. But the idea that I’m going to go to my forum place and find the forums on a lot of topics, rather than I’m going to go to my skiing place and find a forum about skiing there, it was just a different way to enter. And people seemed to find it more natural to go to the topic first and then the functionality of a forum from that, if that makes sense. And the folks with their skiing guide website really, really wanted to own that discussion about skiing. So rec.skiing basically got siphoned off to somebody’s website like onthesnow.com.

Aaron Dinin:

So I don’t know about you, but I find this kind of story extremely fascinating. What Steve is describing here is basically a critical fork in the road of history. Specifically, it represents two completely different directions the online world could have gone. In one direction, you had the Usenet version of online connectivity. It was a version where all the information and knowledge accumulated online, lived as a sort of amalgamation of contributions from everyone. So if you wanted say, tips on good pizza places to eat at in Sheboygan, Wisconsin, you’d find that information stored within a news group, perhaps alt.restaurants.wisconsin.sheboygan.pizza, where people had been posting their opinions and nobody really owned that news group. It was just something that existed in the ether as a public good.

In the other direction, and where we found ourselves, is what we’ve got with the worldwide web version of online connectivity. It’s the version where finding a good pizza place in Sheboygan means going to a review website like Yelp, which is a website owned by a big company, trying to monetize off of you and your guests in order to help you find a delicious slice. And once you’re there, you search for pizza in Sheboygan, and then you filter the results that may or may not be influenced by the owner of those results and that owners need to make money.

To be clear, I’m not suggesting the Usenet version of online connectivity is somehow more pure. In fact, I guarantee if we lived in a world dominated by Usenet rather than the web, plenty of people would have figured out how to manipulate the kinds of information we find on it for the purposes of their own monetary gain. Heck, Steve and Deja are pretty much an example of one of them, right? It just so happens, the strategy he was using to monetize, which was selling advertising, wasn’t as mature as it needed to be.

Steve Madere:

The vast majority of advertising at the time was going through advertising agencies that were trying to prevent the transition to digital media. And so, we get up to a certain point and then just hit a brick wall once you started getting into the larger spenders on advertising, I can go back in time and point at certain accounts that we had and said, ha, there you go. One of our first to advertisers was Jeff Bezos. Personally. He may even remember that if you ever get a chance to talk to him and say, where was the first internet website you were advertised on?

Aaron Dinin:

All right. If I ever get Jeff Bezos here on Web Masters, I’ll be sure to add that to the list of questions I ask him. So as far as Deja, though, was the struggle to get big advertisers the reason it ultimately didn’t last? Maybe because Usenet was too small or something like that.

Steve Madere:

Actually, things were moving along pretty nicely. And we filed for an IPO. We had tens of millions of revenue. We had over $80-million in venture capital and then commitment for another 20 or 30 million in mezzanine financing to get through the IPO. Then, the dot bomb happened. And it was just like this panic run for the exits. All internet advertising is worthless stampede by both investment banks and Silicon Valley’s venture capital companies, et cetera. Anyway, so we had filed for an IPO and then a couple of months later, the dot bomb happened and then it just went deeper and deeper. And then our investors, most of the money that they had put in, we were still holding as cash. It was to fund growth. And so they decided, you know what? Everything’s about to fall apart, so let’s just get our cash back out. And they sold what was left of it to Google. So …

Aaron Dinin:

So it wasn’t the idea didn’t have potential. Even as a search engine for Usenet, as opposed to the web, I guess it sounds like it was more a VC decision to abandon ship, so to speak. You could have monetized and expanded to other things, including the web, if you’d only been given more time. Am I understanding that correctly?

Steve Madere:

I should have insisted that our investors pony up for a more speculative plan. They were constantly trying to get us to prove things that were really, really obvious to Google’s investors. And Google’s investors just ponied up for it because it was really obvious to them. But the investors that we had were like, can you prove anybody’s ever willing to pay for internet advertising? Really, seriously? “We won’t invest another dime until you’ve got more” than X amount of revenue in a month. And at that point, it was really clear what was going on was this early adopter issue. So Google managed to get plenty of funding prior to them being able to get that much advertising revenue. So I really should have pursued some more visionary VCs, as it were, people who could look past the end of the week.

Aaron Dinin:

And looking back on all this now and thinking about what could have been, I imagine selling to Google for pennies on the dollar wasn’t an outcome you’re particularly happy with, is that fair to say?

Steve Madere:

I wasn’t thrilled. Yeah. I was the founder of the company and owned a sizable chunk of common stock. I owned more of it than anybody, but a VC. And when you’ve taken VC money and then you end up liquidating, common stock becomes completely worthless and meaningless. So I didn’t even get any of the Google stock. That was unfortunate.

Aaron Dinin:

Yeah, sorry. That seems like a less than ideal outcome.

Steve Madere:

Yeah, that’s all right. No. It was a tremendous learning experience for me. I basically got a school of hard knocks MBA, so I had a ton of fun.

Aaron Dinin:

And out of curiosity, why do you think Google bought it? And what did they ultimately do with Deja?

Steve Madere:

Well, Google’s mission at the time was to archive all knowledge and make it all searchable. And Dejanews had the only complete archive of Usenet. So they wanted to get that. And they turned it into Google Groups, which they continued to support as an important strategic initiative, whatever, for six or seven years before they finally decided that everything had gone so much to web forums that there was just no point in trying to keep Usenet, as it were, going.

Aaron Dinin:

So you mentioned having a lot of fun. I guess, even though this thing you built that could have been huge, ultimately failed. Why was it so fun? What was such a great experience about it?

Steve Madere:

Well, I feel like I actually participated in the revolution. I grew up in the 1970s and I have older brothers and sisters, and I always felt like I missed out on the world changing events of the 1960s. History happened then. I wasn’t there. And then when I was in graduate school, some of my friends were traveling in Europe during the fall of the Berlin Wall and they got to go and see the Wall go down and then go see the Velvet Revolution happen in Czechoslovakia and stuff like that. And I was working on my Physics graduate degree and I was stuck in a lab. Did I miss out on history again? But, in the 1990s, I actually got to make some of it. So I was pretty psyched about that.

Aaron Dinin:

Making history. That’s exactly what we’re talking about here. As we all know, the history of entrepreneurship and the companies that go on to dominate our world isn’t really a history of multi-billion dollar winners. Entrepreneurial history is a lot more complex than that. And the entrepreneurs building companies have to face that complexity every single day in the decisions they make. They have to choose what markets and what opportunities to pursue. Sometimes they choose correctly. And as we heard in this episode, sometimes they don’t. That’s what happened with Steve Madere and Deja.

He chose to pursue an opportunity he saw around Usenet, and in the moment, it certainly seemed like the right opportunity. Ultimately, however, that wasn’t the right choice. But as Steve’s attitude demonstrates, that’s okay. He learned a ton and had a lot of fun along the way. Hard to argue with that outcome. In fact, I hope it’s the exact same outcome you’ve gotten from listening to this episode of Web Master. If it was, or even if it wasn’t, I hope you’ll take a moment to head on over to your favorite podcasting app in order to Like, Share, Rate, Review, whatever you can do to show your appreciation for Web Masters and for Steve being willing to join us to share his story.

If you’ve got any thoughts or comments on the episode, remember you can find us over on Twitter. We are @WebMastersPod. I’m also on Twitter. So you can shoot me a message there. I’m @aarondinin That’s: a-a-r-o-n, d-i-n-i-n. And if you’d like more content about startups, tech, business and entrepreneurship, be sure to check out my website. It’s: aarondinin.com.

Quick thanks to our audio engineer, Ryan Higgs, for helping me bring together this episode and a thanks to our sponsor, Latona’s. Don’t forget to check out latonas.com if you’re interested in buying or selling an internet business. And if you’re interested in more episodes of Web Masters, you can check out our growing archive. You can also subscribe so you get the next episode as soon as it’s released. We’ll have that for you very soon, Until then, well, it’s time for me to sign off.