Your personal private Google: how Mem plans to reinvent note-taking apps with AI

In the summer season of 2019, Kevin Moody and Dennis Xu began assembly with buyers to pitch their new app. They had this large concept about reshaping the way in which customers’ private info strikes across the web, coalescing all their knowledge right into a single instrument in a method that would truly work for them. But they shortly bumped into an issue: all of their mock-ups and descriptions made it look like they have been constructing a note-taking app. And even in these hazy early days of product growth — earlier than they’d a prototype, a design, even a reputation — they have been crystal clear that this may not be a note-taking app.

Instead, the founders wished to create one thing a lot larger. It would embody your entire notes but in addition your pursuits, your viewing historical past, your works-in-progress. “Imagine if you had a Google search bar but for all nonpublic information,” Xu says. “For every piece of information that was uniquely relevant to you.” 

That’s what Moody and Xu have been truly making an attempt to construct. So they saved tweaking the strategy till it made sense. At one level, their app was going to be known as NSFW, a half-joke that stood for “Notes and Search for Work,” and for some time, it was known as Supernote. But after a number of conferences and months, they ultimately landed on the identify “Mem.” Like Memex, a long-imagined system that people might use to retailer their complete reminiscence. Or like, effectively, reminiscence.

Either method, it’s not a note-taking app. It’s extra like a protocol for personal info, a approach to pipe in all the pieces that issues to you — your e mail, your calendar occasions, your airline confirmations, your assembly notes, that concept you had on the practice this morning — after which mechanically arrange and make sense of all of it. More importantly, it’s meant to make use of cutting-edge AI to provide all that info again to you at precisely the best time and in precisely the best place.

The most bold model of the product would require an enormous quantity of labor and loads of know-how that both doesn’t exist or solely barely works. It would require partnerships with large tech companies that could be reticent to share their knowledge, belief from customers who must compile all of it collectively, and a consumer expertise that’s infinitely versatile with out being too sophisticated or overwhelming. But if Mem can accomplish its objectives and even come remotely shut, it might develop into essentially the most highly effective productiveness instrument on the planet. It might perhaps even assist shift the way in which the web works.

“There’s all this data in the world about us,” Xu says, “and none of it we can actually use, right? It’s all trapped inside of Google’s servers or Facebook’s servers, Netflix, YouTube, all that. What we want to do is actually put that in your hands.”

So far, Mem is… largely a note-taking app. It’s blisteringly quick and intentionally sparse — largely only a timeline of each mem (the corporate’s parlance for a person be aware) you’ve ever created or considered, with a number of easy methods to categorize and arrange them. It does duties and tags, however a full-featured challenge supervisor or Second Brain system this isn’t.

But if you happen to look rigorously, the app already comprises a number of indicators of the place Mem is headed: a instrument known as Writer that may truly generate info for you, primarily based on each its information of the general public web and your private info; AI options that summarize tweet threads for you; a sidebar that mechanically shows mems associated to what you’re engaged on.

All this nonetheless barely scratches the floor of what Mem desires to do and might want to do to be greater than a note-taking app. The competitors might be steep, too: Dropbox lately acquired Command E to energy cross-platform doc search, Neeva is constructing private search proper subsequent to net search, whereas Evernote’s Chrome extension helps you to search your notes by way of Google. Google’s even tried to construct this itself, although it gave up on Google Desktop when it turned clear cloud storage was the longer term. As generative AI will get extra highly effective and cheaper, each app you employ will discover a approach to incorporate it.

The level is, “a search engine for your private stuff” isn’t a brand new and even significantly novel concept. But thanks to large latest developments in AI and a generational shift in how we handle our digital lives, the time is ripe for somebody to construct it.

There’s no lightbulb second on the founding of Mem, extra a sequence of cobbled-together concepts over time. Moody and Xu have been Stanford classmates and had talked about working collectively whereas Moody was at Google and Xu at Yelp. But Moody does bear in mind one particular aha second from 2018 when he was at a restaurant in Oregon. “I was like, ‘Shit, I think I’ve been here before, and I forget what I ordered,’” he says. He couldn’t bear in mind if he ought to reorder what he bought final time or was doubtlessly about to make the identical menu mistake once more. This instantly felt like a solvable drawback. “I wanted to be able to come back to the same restaurant,” he says, “and a note would just pop up and say, ‘The seared ahi, 10/10; the meatball sub, send it back before it even gets to your table.’” He wrote that concept down in Apple Notes and by no means fairly forgot about it.

From the very starting, Mem’s founders knew they wanted an enormous quantity of knowledge to get their imaginative and prescient off the bottom. (Google doesn’t work if there’s no web to crawl, proper?) So the Mem app was constructed with straightforward seize in thoughts, which meant making the app quick to open and quick to sort into. Before they’d a cellular app, they constructed a method for customers to textual content notes to Mem by way of SMS or WhatsApp. That proved to be considered one of Mem’s hottest options. Even within the early days, when Mem didn’t do a lot, it was nonetheless one of many best instruments for simply shortly jotting one thing down.

But the dream isn’t fast notes. The dream is for all of your data and all the pieces about you to only seem in Mem, the place it may be sorted and searched and made helpful in your behalf. If a instrument might entry your Netflix historical past and your Spotify playlists, simply to call one instance, it might study rather a lot about your style. So the staff started early on to seek for even larger sources of non-public info, ones that don’t require customers to take their very own notes. 

It began with the most important participant of all: Google. After all, if you happen to wished a single supply of an individual’s necessary on-line info, their inbox is an efficient place to start out. Mem has been working with Google for months to let customers hook up with their Gmail and permit Mem to index all of their emails since time immemorial. They’re nonetheless engaged on the implementation, however the aim, Xu says, is to “essentially turn all of that unstructured information into something that is really easy to use for you.” After that, Mem might want to do one thing related for, effectively, each different platform on the web.

There’s a superb probability it’s unimaginable to deliver the complete web’s information of you right into a single place — even Xu and Moody acknowledge that the majority corporations don’t have any incentive to provide Mem or anybody else entry to the information they’ve. And even when it might probably pull it off, Mem would immediately develop into a safety nightmare. Xu and Moody say they’ve invested in platform safety because the starting and that they take their safety duty significantly. “If we succeed even close to the degree we want to, we’re going to have the most valuable data in the world,” Xu says, “and that’s going to put a huge target on our back.” In the long term, they’re curious about utilizing decentralized platforms to keep away from having a single level of failure. Both co-founders say they’ll by no means promote or in any other case abuse consumer knowledge. Still, to do what they plan would require an unprecedented quantity of belief from customers at a time when belief in digital platforms appears awfully laborious to come back by.

Mem’s finest — perhaps solely — probability is to make the tradeoff value it. 

A helpful query to ask about any web service is, what are they doing with all of your knowledge? Often, it’s focused adverts, that are of specious worth. Other occasions, it’s to make higher suggestions for stuff to look at or purchase or hearken to. In Mem’s case, it thinks that by gathering and mixing all of your info, it might probably each enable you discover all that knowledge and make it rather more helpful.

In our very first dialog in 2020, Moody off-handedly asks a query that hinted at issues to come back: why doesn’t Google Docs know what’s in your docs? Docs can search by way of the phrases, positive, nevertheless it doesn’t perceive what your doc is about. It can’t generate a desk of contents or a one-paragraph abstract; it might probably’t let you know which of your different docs may be related to the subject at hand. (Ironically, within the months since, Google has truly begun to roll out a few of these options. Moody was onto one thing.) It’s not simply Google, both, he says. Too many notes apps, doc holders, and collaboration instruments are simply “dumb containers of information.” Mem aspires to be greater than that.

Mem’s actual principle of the longer term begins with transformers. Not the robots-in-disguise type — the machine-learning type. It begins with a whitepaper from 2017 known as “Attention is All You Need,” written by a number of Google researchers, that modified the way in which folks take into consideration AI. 

The quick model of an extended paper goes one thing like this: Using transformers, AI engines may be educated to grasp full sentences, paragraphs, even complete paperwork at a time. That provides these engines rather more context with which to deduce that means — transformers enable computer systems to determine whether or not you wrote “bank” that means the home of cash or the aspect of a river or whether or not the tone of a narrative is comfortable or unhappy or someplace in between. Transformers are altering the way in which the entire tech business thinks about language processing. “This has been an incredible breakthrough,” says Zoubin Ghahramani, a distinguished researcher at Google, “that I feel like has enabled a lot of these other interesting language models to flourish over the last five years.” 

The method Mem sees it, transformers flip the entire mannequin of data administration on its head. If the pc can deeply perceive and arrange your stuff, it means you don’t must. For so long as digital note-taking and data storage instruments have existed, they’ve required customers to rigorously are inclined to them. “It takes a lot of time and effort to organize everything,” Xu says, “and keep everything in the right place and make sure you have the right tags. Sometimes it’s just impossible because you don’t realize what connections are going to form and the ways you’re going to want to use a certain piece of information until you get new information.” 

As he explains this to me, Xu shares his Mem display screen over Google Meet and opens up a mem with a Twitter thread he’s writing in regards to the loss of life of folders and digital group methods. As he sorts, an automatic record of associated mems seems in the best sidebar: a number of tweets he’s collected as regards to info storage, an article from The Verge about how younger web customers don’t see the web by way of recordsdata and folders anymore, a number of mems that appeared like different drafts of the identical thread, an early manifesto Moody wrote about Mem’s plans. “If a human were to read this,” Xu says as he scrolls by way of the record, “you’d understand what it’s about, right? And what’s surfacing here is not only things I’ve written in the past; it’s things that Kevin has shared with the team, mission statements — you can create this kind of hive mind.” 

This is Mem X, Mem’s most superior auto-organizational instrument. Mem X is how Mem turns into a “self-organizing workspace,” the time period Xu and Moody have come to make use of to explain the platform. (It’s additionally how Mem plans to earn a living: the essential app might be free, the corporate has mentioned, however superior options like Mem X and Writer will come at a value.) Through a partnership with OpenAI, Mem has begun ingesting large quantities of consumer knowledge and discovering connections with none consumer enter. 

Mem It for Twitter is one other instance: Mem can seize a Twitter thread and generate a abstract of its contents whereas additionally recommending related threads from different customers. Going ahead, Mem desires to do the identical for each URL you save to the app — not simply perceive the URL and even the phrases within the title however all the pieces on the web page and what it’s about. Pair that with information about your historical past, habits, and style, and Mem may also develop into your personal private information to the web.

The tech is way from excellent: Xu, at one level, searches for “transformers” to show that Mem X understands the distinction between language fashions and Optimus Prime, and certainly Optimus Prime crept into the search outcomes. When he looked for my identify — which ought to deliver up all of the conferences we’d had collectively, all of the emails I’d despatched, and extra — it solely introduced up a number of outcomes as a result of I’ve a number of e mail aliases. But Mem X drew sufficient unexpectedly right connections that the worth already outweighs the imperfections. Mem’s AI isn’t excellent, and neither are the basics it’s primarily based on. But they each already work.

Just final week, effectively over a yr after our first dialog, Xu shares his display screen with me on Google Meet and reveals me one thing the staff simply completed — essentially the most futuristic factor Mem has constructed but. It’s known as Mem Writer, and it pairs your notes with a generative AI to show note-taking right into a collaboration between you and Mem’s AI.

To display, Xu begins to write down what would develop into a press launch asserting Mem’s new $23.5 million spherical of funding, which OpenAI is main after seeing what Mem can do. Then he invokes Writer with a slash command and writes, “link to Mem X demo video.” With no additional directions, Mem’s AI goes into Xu’s notes, finds the Loom video with that title, and pastes the hyperlink in place.

In one other be aware, Xu had written that he wished to order some books about giant language fashions and generative AI. Then he writes, “Here’s a list of 10 books that I’m considering:” and has Writer fill out the bulleted record with books like Deep Learning and The Master Algorithm. With one other immediate, Writer fetches Amazon hyperlinks for all 10 books. Another immediate brings in AI-generated summaries of every one.

Then, clearly excited by his personal progress, Xu continues. “I say, ‘Okay, which of the books above are most similar to books I have read before?’” Xu retains notes on all the pieces he reads in Mem, due to course he does — he’s the co-CEO. Writer thinks for a minute and spits again a solution: due to two books he’d learn prior to now, “it seems you might appreciate The Book of Why or The Master Algorithm as they both take a broad, conceptual view of their respective subjects.’”

Again, not all of Xu’s demos work. But a few of them do.

If you go together with Mem’s imaginative and prescient for some time, you get to a world by which Mem virtually turns into web infrastructure. When the corporate raised $5.6 million from Andreessen Horowitz and others in 2021, a16z associate David Ulevitch compared Mem’s potential to the all-knowing Jarvis from Iron Man. And as soon as Mem is aware of all the pieces you do and see and like, it might additionally provide a approach to share that knowledge with others. “Imagine, let’s say, three years down the line,” Xu says, “there’s a ‘Sign in with Mem’ button, right? And you sign into this product called Newflix, and Newflix asks you, ‘Can we access your list of favorite movie genres?’ And you say, ‘Sure.’” He compares Mem to WeChat in that sense — that it’s not simply an app however a part of the material of customers’ on-line lives. 

The app is nothing near that now, in fact, however Mem’s concept of a self-organizing workspace already works surprisingly effectively. I’ve been utilizing the app on and off for greater than a yr and discover myself regularly coming again to Mem as a result of the app is so quick and so good at exhibiting me stuff I forgot I’d even saved that I’ve discovered to only dump all the pieces in there and belief that I’ll have the ability to discover it later. The app is hardly a triumph of design, and the cellular app remains to be very a lot a minimal viable product, nevertheless it’s quick and dependable.

Mem’s ambition appears to develop on a regular basis — the founders often discuss shifting the way in which the web works, connecting platforms and reintegrating methods which were systematically siloed off from each other, perhaps in the future being a menace to Google itself. But even when they will’t take over the web and reinvent knowledge possession, Xu and Moody are fairly positive they will construct a search engine that is aware of your PreCheck quantity, all about your favourite films, and whether or not it’s best to skip the ahi. And they’re fairly positive that’d make your life simpler.