Startup Expect Labs has launched its MindMeld app after months of being in private beta. A crude but quick way to describe it is: Google Now meets Skype. Expect Labs, founded by Tim Tuttle, describes it as a "voice assistant." But that doesn't really do it justice.
Many bloggers and tech sites are reviewing MindMeld. In a way that misses the bigger picture. The app is really a "technology showcase" or demo for something larger and more forward looking. Expect Labs, which charges $4 for the app, doesn't see MindMeld as a money maker and isn't staking its future on the success (or failure) of the app.
First, here's what MindMeld does: it listens to your conversation, with one or several people, and in real time shows you pages and websites that are relevant to the discussion. The sites and data are coming from various APIs and the internet broadly. If you and your friends are talking about going to New York on vacation, for example, it will start showing hotels, restaurants and things to do based on the specifics of your conversation.
The key challenge here is filtering "signal" from "noise" and finding relevant pages and sites. Expect Labs' CEO Tim Tuttle says that the technology has significantly improved over time and the app has changed somewhat from its inception to its launch today. For example, it used to listen to the entire conversation. However now it will pause and users are required to initiate "searching" via an "OK MindMeld" wake up phrase.
The underlying technology seeks to deliver a better search and discovery experience on devices where the keyboard isn't particularly useful or there's no keyboard. There are myriad inputs into "search results" (anticipatory search results): time of day, location and "context" broadly speaking. If you sign in with Facebook it also grabs other information about you as another relevance input.
Expect Labs' technology, while imperfect, is really the fulfillment of the vision behind Google Now: real-time, useful information that dynamically changes based on context. MindMeld is the "1.0" expression of that vision. Speech recognition is from Nuance but the natural language understanding is Expect Labs' own technology.
There are a number of enterprise use cases in development; and one can see this technology being incorporated into a wide range of general and vertical applications. Google Ventures is an investor, as is Intel. Those are two potential buyers of the company.
The technology is impressive and the major practical question for Expect Labs will be where to focus and how to fully express what the technology can do in a commercial context.
Roughly three years ago Steve Jobs opined that search wasn't as central to the mobile user experience as it is on the PC. That sentiment elicited dismissals as naive or self-serving and was generally disputed. This is what Jobs said verbatim:
On the desktop search is where it’s at; that’s where the money is. But on a mobile device search hasn’t happened. Search is not where it’s at, people are not searching on a mobile device like they do on the desktop.
It turns out that when you consider what he actually said, Jobs was exactly right.
Various surveys have found that search is widely used on smartphones. But it's not used as often or as centrally as on the PC. Indeed, search is a more occasional or peripheral experience on smartphones (especially the iPhone), whereas people search many times daily on the PC.
Earlier today Consumer Intelligence Research Partners (CIRP) released survey data about most frequently used mobile apps among US smartphone owners. The survey measured frequency not reach. This is very important to understand about the data. The firm asked mobile users to identify their "three most frequently used [mobile] apps."
CIRP found that Facebook was the leading and most frequently used mobile app. That was followed by Twitter, Candy Crush and Instagram. The surprise is how low Google Search and Google Maps rank on the list.
Google Maps is #12 and Google (the search engine) is #10. We don't get an analysis of usage by platform (i.e., iOS vs. Android). However I suspect we'd see different rankings on the two platforms, with Google doing better among Android users given search's prominence on the Android OS.
It's unclear how large the sample in this survey was and so we can't tell how reliable these data are. In addition these are self-reported data and not behavioral or traffic data. People often report one thing and do something else.
Having said all that, these data strongly argue that what Jobs said is accurate: "People are not searching on a mobile device like they do on the desktop." Although this has been written about at length in the past, if accurate, this more modest mobile search frequency represents an obvious problem for Google as migration from PCs to tablets and smartphones continues.
Last week ShopKick introduced "shopBeacon," which uses Bluetooth low energy (BLE) indoor positioning technology. The company is testing it with Macy's, which has also independently been using indoor location for some time (mainly leveraging WiFi) to enhance its in-store app experience for customers. (See ShopKick demo video.)
ShopKick's adoption of iBeacon is an important move to insert the company back into the in-store shopping conversation. It had been an early pioneer in mobile loyalty, seeking to help retailers drive consumers into stores. But as indoor location has gained momentum ShopKick has largely been on the sidelines -- until now.
ShopKick has a wide range of brands and national retail partners, including Target, BestBuy, Sports Authority and JCPenneys. The company seeks to serve retailers but also "own the customer relationship." Accordingly there's some tension between working with ShopKick and providing a direct indoor-location experience, as Macy's does through its app.
A less-well-known company seeking to do something very similar for retailers is Swirl. Swirl has both a consumer-facing multi-retailer app but also powers the indoor experience for retailer apps through an SDK. Timberland is the company's best-known partner. ShopKick is now also an indoor-location enabler with its shopBeacon BLE beacons.
Apple itself is going to implement iBeacon in its own stores. There are a range of obvious and secondary use cases, including providing enhanced product information and notifications about Genius Bar appointments. Beyond an improved in-store experience, Apple hopes to boost sales through iBeacon. The product can also be used to support in-store mobile payments (see, PayPal Beacon).
It's well established that a majority of consumers have used smartphones in store for research purposes and many are interested in indoor/in-store information. However recent research from ISACA suggests that retailers will need to be judicious about how they use in-store notifications and personalization and not become too "pushy" in trying to upsell and cross-sell consumers.
Another challenge of sorts for retailers with indoor location is the fact that majorities of smartphone shoppers use retailer mobile websites. Indoor-location features are much harder to deliver via websites. Smaller numbers of consumers use retailer apps. This makes sense because apps are typically downloaded and used by a store's most loyal customers, which represent a minority of overall store shoppers.
According to NPD survey data, 71% of smartphone owners access retail websites but only 57% use apps. Many of those apps fall into disuse shortly after they're downloaded. In addition, the survey found that a majority of smartphone shopping-related research was done at home and not on the go, suggesting "that engagement on their smartphone is more of an alternative for online shopping rather than a showrooming tool."
Accordingly in-store information directed at enhancing the customer experience is a way to make apps more relevant and engaging. But as the ISACA study indicates retailers (or mall and venue owners) will need to develop information, content and indoor experiences for customers that are informational and not merely about trying to sell things.
This is a complicated arena for retailers and would-be providers of indoor location and marketing. Experimentation and testing are necessary to determine what's going to "work" for consumers, vendors and venue owners. Macy's is very smart and to be applauded for "getting out in front" of the issue and trying things, notwithstanding the potential exposure to "indoor surveillance" criticisms.
The Future of Privacy Forum's Jules Polonetsky was one of the featured speakers at the inaugural Place Conference. He spoke about indoor location and privacy with Jennifer King from UC Berkeley. We alloted 30 minutes for the discussion but could have easily spent an hour on it.
Privacy is the 800 pound gorilla of indoor location and the issue that challenges and potentially threatens its roll out. Ever since the negative publicity and coverage suffered by Nordstrom retailers have been scared to death of talking about what they're doing with indoor location -- despite the fact that consumers stand to benefit greatly through these innovations.
Hoping to head off regulatory intervention and preempt more ill-informed coverage and negative publicity, Polonetsky's Future of Privacy Forum (and Senator Charles Schumer of New York) announced a code of conduct that will govern indoor analytics and seek to protect consumer privacy.
The companies involved include Euclid Analytics, iInside, Mexia Interactive, Nomi, SOLOMO, Radius Networks, Brickstream and Turnstyle Solutions. Euclide and iInside spoke at the Place Conference.
This list doesn't include all the companies involved in indoor analytics (e.g., Retail Next) but the rest will adopt and abide by the rules announced today. And retailers will follow them. Basically the new rules require clear disclosures, enable consumers to opt-out of indoor tracking, make any tracking anonymous and prevent the misuse of information gained by venue owners.
People always forget that much more intrusive closed-circuit video cameras have been in retail environments for more than a generation.
As our panel on indoor analytics pointed out most of the data aggregated and anonymously captured by retailers will translate into in-store improvements, from staffing to store layouts. However consumers need to be educated about all of this given how new and little understood it is. This is where retailers need to step up (rather than cower) and help consumers understand why and how indoor location will benefit them.
Hopefully this new code of conduct will enable them do that with confidence.
Below is the full text of the press release:
New York, NY – U.S. Senator Charles E. Schumer, The Future of Privacy Forum (FPF) and a group of leading location analytics companies – including Euclid, iInside (a WirelessWERX company), Mexia Interactive, SOLOMO, Radius Networks, Brickstream and Turnstyle Solutions – today announced that they have agreed to a Code of Conduct to promote consumer privacy and responsible data use for retail location analytics. The companies responded to privacy concerns raised by Senator Schumer and the FPF about the use of this new technology. The code of conduct includes in-store posted signs that alert shoppers that tracking technology is being used, and instructions for how to opt out.
“This is a significant step forward in the quest for consumer privacy,” said Schumer. “This agreement shows that technology companies, retailers, and consumer advocates can work together in the best interest of the consumer. There is still much more work to be done and I will continue to push for privacy rights to be respected and strengthened, but this represents real progress and I thank the Future Privacy Forum and these tech companies for their hard work hammering out this agreement.
“Today, location analytics companies have introduced a comprehensive code to ensure they have data protection standards in place to de-identify data, to provide consumers with effective choices to not be tracked and to explain to consumers the purposes for which data is being used,” said Jules Polonetsky, executive Director of the Future of Privacy Forum. “These standards ensure that consumers understand the benefit of the bargain and have choices about how their information is used while allowing technology to continue to improve the shopping experience. As we quickly approach the holiday shopping season, this is not only the right move – but a timely one as well, adding a layer of trust, choice and transparency onto a shopping experience that in 2013 is more mobile and hi-tech than ever before.”
In July, Schumer warned that major national retail chains were testing technology that would allow them to automatically track shoppers’ location through stores. Following this warning, FPF worked with the technology companies to develop a Code to ensure that appropriate privacy controls are in place as retailers seek to improve the consumer shopping experience. These technology companies use mobile device Wi-Fi or Bluetooth MAC addresses to develop aggregate reports for retailers.
The Code puts guidelines in place to create best data practices that will provide transparency and choice for consumers. The Code calls for the display of conspicuous signage by retailers and for a central opt-out site for consumers.
"We are just beginning to see the possibilities that in-store analytics can bring to shoppers and to retailers, and yet, as with any new technology, there is the chance for confusion about the intent and possible implications of such technology,” said Steve Jeffery, CEO, Brickstream. “We applaud the Future of Privacy Forum for taking the lead in bringing retailers and technology providers together to address these important issues.”
“We would like to thank Senator Schumer for his leadership on this issue,” said Will Smith, CEO, Euclid. "Privacy has always been a priority as we've designed and built our services, and we have been working diligently with FPF to release best practices for the retail analytics industry as a whole.”
"iInside and industry partners have made it a top priority to assure that consumers are well-informed and their personal privacy and identity are protected. The newly announced code is a major step forward in establishing and communicating clear and concise standards across our industry," said Jim Riesenbach, CEO, iInside Inc.
“The release of a Code of Conduct to guide industry practice ensures that businesses and retailers are able to enhance their customers’ experience without compromising their privacy,” said Glenn Tinley, President & Founder, Mexia Interactive. “Business and consumers also can be assured that a company listed on the SmartStorePrivacy.com website has committed to following the code.”
"Proximity and location technology is evolving rapidly, and we want to make sure it’s deployed in an open, responsible and trustworthy manner. The retail location analytics Code of Conduct is a solid step in the right direction," said Marc Wallace, Co-Founder & CEO, Radius Networks, Inc.
“SOLOMO sees privacy as an opportunity for retailers to build trust with customers,” said Liz Eversoll, CEO, SOLOMO. “We’ve collaborated to develop the Code of Conduct to ensure transparency and empowerment for retail customers. Indoor location technology will offer customers new in-store experiences, special deals, and localized services as retailers introduce it in their stores. Everyone wins.”“Turnstyle Solutions is pleased to partner with the Future Privacy Forum in the development of this Code of Conduct. We are confident the code lays the foundation necessary to protect sensitive consumer information, while offering retailers and consumers services that enhance their shopping experience," said Devon Wright, Co-Founder, Turnstyle Solutions.
Place 2013 brought together the entire spectrum of companies building the indoor location ecosystem. Retailers, technology vendors, mobile developers, data providers, advertisers, agencies, and investors attended this unique, one-day event at the Palace Hotel in San Francisco and was the first-of-its-kind anywhere.
8:45 AM - 9:00 AM
The Consumer Foundations of Place-Based Marketing - The majority of smartphone owners are already using their devices in stores to find product and price information, as well as coupons. Opus Research will present proprietary findings on in-store behavior, privacy attitudes and consumer receptiveness to indoor promotions.
Speaker: Greg Sterling, Senior Analyst, Opus Research
View slides from this presentation
9:00 AM - 9:45 AM
The State of Indoor Location - For the past several years online mapping giants and technology providers have been laying the groundwork for indoor location. What is the current state of the infrastructure? What technologies are already deployed and how accurate are they? What indoor consumer and advertiser scenarios are possible today and what might be possible within three years?
Joseph Leigh, Head of Venue Maps, Nokia
Leslie Presutti, Mobile, Location and Computing Business Unit, Qualcomm Atheros
Zack Sterngold, VP of Americas, Boingo Wireless
Avinash Joshi, Chief Technologist, Wireless LAN Group, Motorola Solutions
9:45 AM - 10:25 AM
Keynote: Why Indoor Location Will Be Bigger than GPS or Maps - The explosion of smartphones with built-in sensors, accelerometers, GPS and WiFi is making indoor positioning not only possible but also inevitable. The emerging indoor opportunity for venue owners, retailers and technology providers is potentially massive. Google’s Don Dodge, an investor and close observer of the space, will explain why he believes indoor location and marketing is going to be huge and potentially larger than GPS and maps.
Speaker: Don Dodge, Developer Advocate, Google
10:45 AM - 11:05 AM
Case Study: Point Inside - Point Inside was one of the early consumer-facing apps in the indoor location space. The company has since shifted its focus to enterprises and enabling retailers to take advantage of indoor location. The company will present a new case study featuring a major home-improvement retailer.
Speaker:Todd Sherman, Chief Marketing Officer, Point Inside
View slides from this presentation
11:05 AM - 11:30 AM
Featured Case Study: Forest City and Path Intelligence - Forest City Enterprises are many years into using mobile device monitoring and advanced indoor analytics to help create a better environment for their shoppers and their retailers. Hear from the project sponsor and partner Path Intelligence on how they have transformed asset management, leasing, and marketing.
Stephanie Shriver-Engdahl, VP, Digital Strategy, Forest City
Cyrus Gilbert-Rolfe, VP, Path Intelligence
View slides from this presentation
11:30 AM - 12:15 PM
Digital Analytics for the Real World - Using a variety of technologies to identify when and where smartphone shoppers are in stores, retailers can now leverage "big data" previously reserved for Internet companies alone. These "real world analytics" hold profound implications for everything from in-store merchandising and staffing to consumer marketing. Leaders in the segment will offer views on opportunities and potential pitfalls for indoor analytics.
Jon Rosen, Executive Vice President, iInside
Will Smith, CEO, Euclid
Alexei Agratchev, Co-Founder, RetailNext
Michael Healander, General Manager, GISi Indoors
1:15 PM - 1:55 PM
Retail Spotlight: Aisle411 & Dick's Sporting Goods - Aisle411 will discuss current retail deployments and their impact on operations, consumer loyalty and marketing. Dick’s Sporting Goods will share how it’s thinking about indoor location, privacy issues and the overall opportunity. And Bob Rosenblatt, former COO of Tommy Hilfiger Group, will outline the intriguing business opportunities for retailers in develop- ing indoor marketing strategies.
Nathan Pettyjohn, Founder & CEO, aisle411
Rafeh Massod, VP, Customer Innovation Technology, Dick's Sporting Goods
Bob Rosenblatt, CEO, Rosenblatt Consulting
View slides from this session from aisle411
1:55 PM - 2:15 PM
Using Store Visits and Data for Advanced Retail Intelligence - Online to offline has been the dominant but largely invisible paradigm of Internet-driven spending. Using mobile to better target and influence store visits is only the beginning. PlaceIQ CEO Duncan McCall will offer a major retail case study fo- cused on measuring store visits after mobile ad exposures. He will also discuss how to connect online, nearby and indoors for a more complete picture of the customer journey.
Speaker:Duncan McCall, Co-Founder & CEO, PlaceIQ
View slides from this presentation
2:15 PM - 3:00 PM
Ad-Tracking to the Point of Sale - Panelists will discuss the current and future use of indoor location as a way to demonstrate ROI and sales lift on a per- campaign basis. What is the current state of the art in matching store visits to ad exposures? And what are the broader implications of connecting online ads and offline data?
Monica Ho, Vice President of Marketing, xAd
David Shim, Founder & CEO, Placed
Ameet Ranadive, Director of Product, Twitter Ads Team
Michael Shevach, SVP Ad Solutions, Retailigence
Duncan McCall, Co-Founder & CEO, PlaceIQ (moderator)
3:20 PM - 3:50 PM
Opt-in or Opt-out: Indoor Location & Consumer Privacy - Indoor location has already gained the attention of members of Congress and been called "troubling." While not everyone agrees about the level of concern, there are obvious consumer privacy issues raised by in-venue smartphone tracking. How should the companies be addressing these issues today and what might regulation require tomorrow?
Jennifer King, School of Information, UC Berkeley
Jules Polonetsky, Executive Director & Co-chairman, Future of Privacy Forum
3:50 PM - 4:10 PM
Case Study: Meridian/Aruba Networks - Meridian, who was recently acquired by Aruba Networks, will offer two indoor case studies, one involving a small business (Powell’s Books in Portland) and another involving a major U.S. apparel and housewares retailer.
Speaker: Jeff Hardison, Vice President, Meridian
View slides from this presentation
4:10 PM - 4:55 PM
Microfencing: Targeting In-Aisle Shoppers - Billions of dollars are spent each year by brands and manufacturers trying to influence consumer buying in stores. A percentage of that money will migrate to indoor digital marketing. What conditions must first exist and what will those brand-consumer interactions look like? The panel will explore these questions as well as the contours of the broader indoor marketing experience.
Neg Norton, President, Local Search Association Ben Smith, CEO, Wanderful Media
Melissa Tait, VP of Technology, Primacy
Erik McMillan, CEO, BrickTrends
Asif Khan, Founder, Location Based Marketing Association (moderator)
4:55 PM - 5:30 PM
Reality Check: Assessing the Indoor Opportunity - The other sessions explored major opportunities (and challenges) of indoor location and marketing. Now it’s time for a fun, yet sober assessment of whether and how soon these scenarios will come to pass. Is there real demand and who will own the “indoor channel”? Where will the "place-based market" be next year, in three years?
Jeremy Lockhorn,VP, Emerging Media, Razorfish
John Gardner, Partner, Nokia Growth Partners
Chandu Thota, Engineering, Google
Wibe Wagemans, IndoorAtlas
One of the questions that we'll be addressing on the "Microfencing" (in-store/in-aisle targeting) panel at Place 2013 is "who will own the indoor channel?" The operating assumption is that the venue owners/retailers will control communications and marketing within their indoor environments. But that may not turn out to be true if retailers aren't careful and quick to embrace indoor location.
An analogy may be the wireless carriers. Once the gatekeepers of all things mobile, they have largely been sidelined and reduced to "commodity" providers of bandwidth. The handset OEMs, platform providers and app developers dominate mobile.
Earlier today Reuters reported that Cisco was "working with Facebook Inc to offer free Wi-Fi Internet access to consumers at public places such as hotels or retail stores using their Facebook log-in. A visitor could check in at a hotel without having to line-up at a front desk by simply signing in via the Facebook application on a smartphone, Cisco said."
Google is also contemplating its own WiFi infrastructure. By the same token some retailers are hesitant or cautious about embracing indoor location. For example, JC Penney decided to eliminate public WiFi to save $7 million a year. In doing so it shut down its indoor location consumer infrastructure. While it will save some money it won't stop showrooming and may deprive JC Penney of an important marketing and customer service capability.
If companies such as Facebook and Google, or other third parties, step in and provide the consumer network, chances are very good that consumers will use the Facebook or Google network rather than the store's. That would likely make the Google and Facebook the new gatekeepers of indoor marketing, giving them a significant advantage over retailers and a stronger position when it comes to selling and delivering indoor advertising and promotions.
Retailers cannot and should not "wait and see" or they may find themselves, like the wireless carriers, on the sidelines of digital marketing activity in their own stores. I could be wrong and there are a number of unknown variables. But retailers stand to lose much if they fail to act.
We'll fully explore these questions at the Place Conference on Tuesday, October 8:
Microfencing: Targeting In-Aisle Shoppers
Billions of dollars are spent each year by brands and manufacturers trying to influence consumer buying in stores. A percentage of that money will migrate to indoor digital marketing. What conditions must first exist and what will those brand-consumer interactions look like? The panel will explore these questions as well as the contours of the broader indoor marketing experience.
I wrote several days ago on my Screenwerk blog about PayPal's new Beacon payments and indoor location initiative. I explained that Apple's decision not to include an NFC chip in the new iPhone means essentially that NFC is marginalized if not dead in the US market. In its place Bluetooth Low Energy (BLE) may become the mainstream alternative to what NFC would have enabled (e.g., mobile payments).
PayPal Beacon relies on BLE. At those businesses where a Beacon device is plugged in PayPal users simply check-in. Beacon identifies them and payment is automatically transferred from the default account. Payment happens “hands free” without a tap, swipe or other app interaction.
Beacon is also PayPal’s entry into indoor location. PayPal will obviously know you’re in a venue and then can do any number of things, including delivering highly specific, indoor marketing messages or ads. PayPal is also making Beacon available to third party developers, who will be able to do similar things accordingly.
Apple's iOS 7, which will be available on September 18 (to older iPhones as well), will permit all eqipped devices to interact with BLE iBeacons in malls, airports, stores and other venues.
Apple acquired indoor mapping company WiFiSlam earlier this year. That was the "wake up call" for many people to take indoor location seriously. One of the first and obvious applications of iBeacon is indoor mapping. But it doesn't stop there.
Just as with PayPal's BLE initiative, iBeacon will enable Apple to move into payments and indoor marketing and allow third party developers to leverage those capabilities. With its more than 600 million credit cards on file I've got to believe that Apple will enter mobile payments eventually and BLE will be the way in all likelihood.
More broadly I suspect that iBeacon will popularize and jumpstart indoor location for a host of third party developers.
For a comprehensive introduction to the indoor location and marketing opportunity, and its broader implications, come to Place 2013.
Last week Placed introduced Placed Attribution, a mobile ads offline tracking solution. The idea is to used Placed's opt-in panel to measure the impact of mobile ad exposures on in-store visits. PlaceIQ has a similar offering using a different methodology.
Capturing the offline impact of digital ads on store visits (and potentially sales) is really a kind of "holy grail" when it comes to conversion tracking. The ratio of online to offline conversions is skewed heavily in the direction of offline. E-commerce is only 5.5% of offline retail and mobile commerce is approaching 10% of e-commerce.
Yet up to half of offline retail spending may not be impacted by digital media and the internet. Clicks are a terrible metric for mobile advertising, and secondary metrics like map views and calls are better but don't capture the entire picture for marketers.
There's a lot more "visibility" on performance when you can start to measure how digital ads impact offline purchase activity. That's the objective of Placed Attribution. Here are the kind of data to be reported: