The biggest mistake businesses make about AI

The Ant(s) and the Grasshopper - Aesop's fable on preparedness. Illustration by Milo Winter (1919)

AI (broadly defined to include data science, machine learning, and cognitive computing) can be confusing for decision makers. It promises great rewards but also great risks. AI is seemingly everywhere but it is also difficult to understand. It is touted as the way of the future but there is little experienced talent available. How does a business navigate this labyrinth? How does a business minimize risk and avoid mistakes?

Through our experience of applying AI within diverse businesses, we find the biggest mistake businesses make about AI to be quite basic: they start thinking about AI too late. This means businesses are often ill-prepared and not set up for success for when the time comes to invest in AI.

“By failing to prepare, you are preparing to fail.” ― Benjamin Franklin 

I say this because most businesses know that AI already impacts or will impact their customers, industry, and themselves. It is well recognized that AI will be incredibly important to the future of the business.

The chart shows expected impact of AI on Offerings. Similar effect is identified for impact on Processes as well. - Image Credit: https://sloanreview.mit.edu/projects/reshaping-business-with-artificial-intelligence/

Yet, it’s difficult to identify the exact implications, the details, of AI’s impact and how to respond and innovate. Lack of knowledge, resource constraints, organizational inertia, and lack of experienced talent compounds this problem where it becomes natural to simply delay “thinking about AI.” Fear of the unknown and of change don’t help! One of the most common ways we work with our customers is to help them work out how AI impacts their business - is it a threat or an opportunity? Typically, it’s both. Then, the next step is to figure out a roadmap to capitalize on the opportunity and address the threats.

Key to success: Prepare

I am not advocating that all businesses must invest in AI right away. Don't need to start hiring new employees or buying new software or services. Instead, I am proposing that companies take the time now, with some urgency, to think about how AI impacts their customers, business, and industry. And then, if necessary, take action to ensure that the business is setup for long-term success in an AI-enabled world.

The biggest danger of delay is that eventually when the business is ready to invest in AI, the necessary prerequisites and key success factors are missing - talent, culture, and especially, data. This further delays “AI development” and poses a significant competitive risk.

By preparing, the business can ensure it’s in control of its future. While this has always been true in business, complacency is especially dangerous when is comes to AI. AI requires data and that takes time.

Must have data

Thinking about AI early on can provide a valuable strategic lead when it comes to collecting data for the future.

Image Credit: The Economist

We live in the "age of data".

Data has always been useful in making critical decisions and driving successful business outcomes. Now, data has become the most valuable commodity in the world because by leveraging AI, data can be used to improve and automate decisions and processes.

This makes data collection critical. But, it can be challenging and can take time to get right: what data should be collected, how should it be stored securely, and what competencies and processes are needed to leverage the data.

Collecting just any data is not enough. Data needs to be differentiated, high quality, labeled, and meaningful enough to be useful. It can be easy to dismiss ethics of data collection but businesses worthy of our trust will take great care of their customers’ data.

There are other challenges too. Data collection often runs into a chicken-and-an-egg problem where you need some data (or AI) to gather other data that you need. This requires developing and launching a “basic” or less-than-ideal solution early on. This is an approach Facebook and Google use quite effectively and we often recommend as well.

There is added urgency because data can become exponentially valuable. Value of data compounds over time. Data often has its own gravity as well: data attracts even more data. For example, collecting data allowed Google to build a better search engine, leading to higher usage. This led to even greater data collection, and so on. A virtuous cycle of data collection, better product, and higher usage developed. This led to a significant competitive advantage and accumulation of value.

The later you start, the farther behind you fall

This exponential economics of data (compounding, gravity, and virtuous cycle) also has a downside: the later you start, the farther behind you fall. If a competitor started six months before you, just as an example, they may effectively have a nine month headstart.

It takes time for a business to learn how to think about AI, build the right culture, gather necessary data, earn trust of customers, and then actually build and launch the AI. Building a data-driven, AI-powered virtuous cycle takes a long time. However, average lifespan of businesses is declining. Businesses have less and less time to adapt to changes that impact their industries. Time, gained through preparation becomes a key competitive advantage.

Image Credit: Microsoft

This is why I believe there is no time like the present for businesses to start thinking about AI. Even if businesses don’t invest in developing or implementing this technology right now, it is critical that they learn to think about this, start painting a vision, identify a roadmap, and at a bare minimum, start collecting the necessary data.

Two years from now, when you are ready to start your AI journey, you don’t want to find out that you should have spent a few hours collecting some data two years ago.

A business doesn’t want to run out of time. 

Don’t make the biggest, easiest, and most common mistake businesses make with AI. Start preparing, now.

This article was also posted on LinkedIn.

Making a Custom Object Detector using Pre-trained Model in Tensorflow

Recent development in computer vision has enabled exciting new technologies like self-driving cars, gesture recognition, and machine vision. The processing power required to create computer vision models was a barrier of entry for those interested in exploring this technology. However, this is no longer the case with pre-trained models today.

Instead of training your own model from scratch, you can build on existing models and fine-tune them for your own purpose without requiring as much computing power.

In this tutorial, we’re going to get our hands dirty and train our own corgi detector using a pre-trained SSD MobileNet V2 model.

1. Installation

This tutorial is based on Anaconda virtual environment with Python 3.6.

1.1 Tensorflow

Install Tensorflow using the following command:

$ pip install tensorflow

If you have a GPU that you can use with Tensorflow:

$ pip install tensorflow-gpu

1.2 Other dependencies

$ pip install pillow Cython lxml jupyter matplotlib

# Install protobuf using Home Brew
$ brew install protobuf

For protobuf installation on other OS, follow the instructions here.

1.3 Clone the Tensorflow models directory

In this tutorial, we're going to use resources in the Tensorflow models directory. Since it does not come with the Tensorflow installation, we need to clone it from their Github repo:

First change into the Tensorflow directory

# For example: ~/anaconda/envs/<your_env_name>/lib/python3.6/site-packages/tensorflow

$ cd <path_to_your_tensorflow_installation>

Clone the tensorflow models repository

$ git clone https://github.com/tensorflow/models.git

From this point on, this directory will be referred to as the models directory

1.4 Setting up the environment

Every time you start a new terminal window to work with the pre-trained models, it is important to compile Protobuf and change your PYTHONPATH.

$ cd <path_to_your_tensorflow_installation>/models/research/

$ protoc object_detection/protos/*.proto --python_out=.

$ export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

Run a quick test to confirm that the Object Detection API is working properly:

$ python object_detection/builders/model_builder_test.py

If the result looks like the following, you're ready to proceed to the next steps!

...............
----------------------------------------------------------------------
Ran 15 tests in 0.123s

OK

To make this tutorial easier to follow along, make sure you have the following folder structure within the models/ directory you just cloned:

models 
    ├── annotations
    |   └── xmls    
    ├── images
    ├── checkpoints
    ├── tf_record
    ├── research
    ...

These folders will be used to store required components for our model as we proceed.

2. Collect images

Data preparation is the most important part of training your own model. Since we're going to train a corgi detector, the first step is gathering pictures of corgis! About 200 of them would be sufficient.

I recommend using google-images-download to download images. It searches Google Images and then download images based on the inputs you provided. In the inputs, you can specify search parameters such as keywords, number of images, image format, image size, and usage rights.

Since we're downloading more than 100 images at a time, we need a chromedriver in the models directory (download here). You could use this sample command to download images. Make sure all your images are in the jpg format:

# From tensorflow/models

$ googleimagesdownload --keywords 'welsh corgi dog' \
    --limit 200 \
    --size medium \
    --chromedriver chromedriver \
    --format jpg

After downloading, save all images to models/images/. To make subsequent processes easier, let's rename the images as numbers (e.g. 1.jpg, 2.jpg) by running the following script:

3. Label your data set

Once you've collected all the images you need, you need to label them manually. There are many packages that serve this purpose. labelImg is a popular choice.

labelImg provides a user-friendly GUI. Plus, it saves label files (.xml) in the Pascal VOC format, which makes subsequent data conversion easier. Here's what a labelled image looks like in labelImg:

corgi_screenshot.png

Double check that every image has a corresponding .xml file and save them in models/annotations/xmls/.

4. Create Label Map (.pbtxt)

Classes need to be listed in the label map. Since we're only detecting corgis, the label map should contain only one item like the following:

Note that id should start from 1, because 0 is a reserved id. Save this file as label_map.pbtxt in models/annotations/

5. Create trainval.txt

trainval.txt is a list of image names without file extensions. Since we have sequential numbers for image names, the list should look like this:

Save this file as trainval.txt in models/annotations/

6. Create TFRecord (.record)

TFRecord is an important data format designed for Tensorflow. (Read more about it here). Before you can train your custom object detector, you must convert your data into the TFRecord format.

Since we need to train as well as validate our model, the data set will be split into training (train.record) and validation sets (val.record). The purpose of training set is straight forward - it is the set of examples the model learns from. The validation set is a set of examples used DURING TRAINING to assess model accuracy.

We're going to use create_tf_record.py to convert our data set into train.record and val.record. Download here and save it to models/research/object_detection/dataset_tools/.

This script is preconfigured to do 70-30 train-val split. Execute it by running:

# From tensorflow/models

$ python research/object_detection/dataset_tools/create_tf_record.py

If the script was successful, train.record and val.record should appear in your models/research/ directory. Move them into the models/tf_record/ directory.

7. Download pre-trained model

There are many pre-trained object detection models available in the model zoo. In order to train them using our custom data set, we need to restore them in Tensorflow using their checkpoints (.ckpt files), which are records of a previous model state.

For this tutorial, we're going to download ssd_mobilenet_v2_coco here and save its model checkpoint files (model.ckpt.meta, model.ckpt.index, model.ckpt.data-00000-of-00001) to our models/checkpoints/ directory.

8. Modify Config (.config) File

Each of the pretrained models has a config file that contains details about the model. To detect our custom class, the config file needs to be modified accordingly.

The config files are included in the models directory you cloned in the very beginning. You could find them in:

models/research/object_detection/samples/configs

In our case, we'll modify the config file for ssd_mobilenet_v2_coco. Make a copy of it first and save it in the models/ directory.

Here are the items we need to change:

  1. Since we're only trying to detect corgis, change num_classes to 1
  2. fine_tune_checkpoint tells the model which checkpoint file to use. Set this to checkpoints/model.ckpt
  3. The model also needs to know where the TFRecord files and label maps are for both training and validation sets. Since our train.record and val.record are saved in tf_record folder, our config should reflect that:
train_input_reader: {
  tf_record_input_reader {
    input_path: "tf_record/train.record"
  }
  label_map_path: "annotations/label_map.pbtxt"
}

eval_input_reader: {
  tf_record_input_reader {
    input_path: "tf_record/val.record"
  }
  label_map_path: "annotations/label_map.pbtxt"
  shuffle: false
  num_readers: 1
}

9. Train

At this point, your models directory should look like this:

models 
    ├── annotations
    |   ├── label_map.pbtxt
    |   ├── trainval.txt
    |   └── xmls
    |       ├── 1.xml
    |       ├── 2.xml
    |       ├── ...
    |
    ├── images
    |   ├── 1.jpg
    |   ├── 2.jpg
    |   ├── ...    
    |
    ├── checkpoints
    |   ├── model.ckpt.data-00000-of-00001
    |   ├── model.ckpt.index
    |   └── model.ckpt.meta
    |
    ├── tf_record
    |   ├── train.record
    |   └── val.record
    |
    ├── research
    |   ├── ...
    ...

If you have successfully completed previous steps, you're ready to start training!

Follow the steps below:

# Change into models directory
$ cd tensorflow/models

# Make directory to store training progress
$ mkdir train

# Make directory to store validation results
$ mkdir eval

# Begin training
$ python research/object_detection/train.py \
    --logtostderr \
    --train_dir=train \
    --pipeline_config_path=ssd_mobilenet_v2_coco.config

Training time varies depending on the computing power of your machine.

10. Evaluation

Evaluation can be run in parallel with training. The eval.py script checks the train directory for progress and evaluate the model based on the most recent checkpoint.

# From the tensorflow/models/ directory
$ python research/object_detection/eval.py \
    --logtostderr \
    --pipeline_config_path=ssd_mobilenet_v2_coco.config \
    --checkpoint_dir=train \
    --eval_dir=eval

You can visualize the model training progress using Tensorboard:

# From .../tensorflow/models directory 

$ tensorboard --logdir=./

11. Model export

Once you finish training your model, you can export your model to be used for inference. If you've been following the folder structure, use the following command:

# From .../tensorflow/models

$ mkdir fine_tuned_model

$ python research/object_detection/export_inference_graph.py \
    --input_type image_tensor \
    --pipeline_config_path ssd_mobilenet_v2_coco.config \
    --trained_checkpoint_prefix  train/model.ckpt-<the_highest_checkpoint_number> \
    --output_directory fine_tuned_model

12. Classify images

Now that you have a model, you can use it to detect corgis in pictures and videos! For the purpose of demonstration, we're going to detect corgis in an image. Before you proceed, pick an image you want to test the model with.

The models directory came with a notebook file (.ipynb) that we can use to get inference with a few tweaks. It is located at models/research/object_detection/object_detection_tutorial.ipynb. Follow the steps below to tweak the notebook:

  1. MODEL_NAME = 'ssd_mobilenet_v2_coco_2018_03_29'
  2. PATH_TO_CKPT = 'path/to/your/frozen_inference_graph.pb'
  3. PATH_TO_LABELS = 'models/annotations/label_map.pbtxt'
  4. NUM_CLASSES = 1
  5. Comment out cell #5 completely (just below Download Model)
  6. Since we're only testing on one image, comment out PATH_TO_TEST_IMAGES_DIR and TEST_IMAGE_PATHS in cell #9 (just below Detection)
  7. In cell #11 (the last cell), remove the for-loop, unindent its content, and add path to your test image:
image_path = 'path/to/image_you_want_to_test.jpg'

After following through the steps, run the notebook and you should see the corgi in your test image highlighted by a bounding box!

denny_found.png

There you have your custom corgi detector! In the next tutorial, I'll be walking you through the set up of real-time object detection on your webcam. Stay tuned!

More details

Tensorflow Object Detection Model Documentation

Why ML/AI development is different from "Traditional" software development

At Gradient Ascent, we often work with software companies to build new features or products powered by Machine Learning (ML) or Artificial Intelligence (AI).

Through our work, it has become evident that developing ML/AI software is different from developing “traditional” software. Recognizing these differences can reduce risks associated with the development of this new generation of software. This is a new, rapidly evolving, and broad area.

In this article we will discuss three key differences and how they impact software development.

#1. AI development is data driven. 

The objective of most ML/AI development is to produce a “model” that generates the output one wants from the provided inputs. For example, Siri’s model takes our speech and turns it into text, and then into an action.

(Image credit: The Economist, https://www.economist.com/printedition/2017-05-06)

 

Majority of ML/AI models are built by using data to train the model. Which is why, it’s no exaggeration to say that data is becoming the defining competitive advantage of the 21st century. With ML/AI development, the training data is effectively the software.

Data is the Software. 

This makes it critical for teams to build competency, tooling, processes, and culture with the goal of building a data-driven organization. Even if a company isn’t applying ML/AI currently, it is critical to get comfortable with collecting, storing, and processing data at scale.

Working with data at scale and in a real-time environment involves novel design trade-offs and requires different architectures. But, this is about more than just “technical” or “engineering” skills. Product managers and designers need to consider how to collect data, design “model feedback loops”, and make rigorous decisions driven by data. Quality assurance needs tools, datasets, and knowledge (for example, statistics) to be able to test models. Serious cross-functional attention needs to be paid to growing regulatory requirements and customer expectations around privacy and security.

This critical need to gather and leverage data also has strategic implications. For example, features that help data collection may need to be prioritized as they enable future innovation. Similarly, “beta” versions of models may need to be deployed to accelerate learning.

Companies that succeed won’t have just a specialized and siloed “data/AI” team but an organizational culture that understands, values, and uses data to build better products that customers love.

#2. AI development is experimental.

While traditional software development can be experimental (rarely), ML/AI model development is inherently experimental and iterative. Trial-and-error is central to any model development, even if the initiative doesn’t require cutting edge research. For example, if I were building a model to predict the risk of a loan default, I’d experiment with a variety of input data and algorithms prior to arriving at a “production-worthy” model. It’s important to note that often, no matter how hard one tries, the model just won’t produce the expected results. In ML/AI development, experiment failure is a routine part of the process.

(Image credit: XKCD, https://xkcd.com/1838/)

 

This experimental nature means the development process is never linear, often unpredictable, and prone to failure. However, traditional software development (especially Scrum and Agile methodologies) are often “optimized” for velocity and predictability.

This divergence has a broad impact within the software development lifecycle. Particularly, extra care is needed in managing roadmaps, timelines, and expectations. Iterative and incremental nature of progress requires more effective communication within internal teams and externally to customers. Broader team members also need to be able to understand this new development cycle so that they can respond to it and provide the necessary feedback.

It also means, talented AI developers have a different mindset from traditional developers. Curiosity, hypothesis-testing mindset, patience, and tolerance for failure become very important skills in this new experimental way of working.

#3. AI development is probabilistic.

Behind all the hype and fear, Data Science, ML, and AI are fundamentally just math. It’s about leveraging computing power, data, and algorithms based-on various branches of math, such as linear algebra, statistics, calculus, etc. The answers generated by most of these systems are numbers, specifically, probabilities. Moreover, the answers in aggregate are never 100% accurate.

However, traditional software development is algorithm- or rule-based. All responses are predictable, repeatable, and unless designed to be probabilistic, never respond with probabilities.

This is a stark contrast in underlying philosophies of how software works.

Most of us are not comfortable with probabilities. It will require experienced designers to convert the output probabilities of these models and turn them into delightful, transparent, trustworthy, insightful, and actionable user experiences.

Similarly, most of us expect software to work 100% of times and provide accurate results. With ML/AI, that’s almost impossible as any interaction with Siri or Alexa would indicate. Traditionally, we would call these “bugs” but with AI, this is “working as designed.”

(Image credit: Wikimedia, https://en.wikipedia.org/wiki/Precision_and_recall)

 

Convincing customers and end-users to use these probabilistic and less-than-perfect systems will be a critical challenge for most organizations. Proper exception handling, effective user interface, and rule-based “guard rails” will be needed for most enterprise-grade software. Marketing and sales teams need to effectively manage expectations. Whereas support agents needs to be able to effectively educate and help customers.

There is a desire to treat these systems as “black boxes” but to build trust with customers will require greater awareness, transparency, and education.

Not all employees will need to be an expert but enough will need to be comfortable with the core concepts (including, math) to be able to design, build, evaluate, explain, and support these systems. 

Key to success: Pragmatism.

ML/AI development doesn’t have to be complex or difficult. Recognizing these differences and embracing the change they bring is a key success factor. Forcing the “old” ways of doing things may not work.

Approach your ML/AI development as a learning exercise. This area is rapidly evolving. In some ways, best practices are still yet to come and not a lot of people have practical experience with this. I will share our “lessons learned” and internal best practices in a future article.

----

I welcome your questions, feedback, and experiences in the comments below.

I would particularly love to hear from you on: How do you approach these issues within your own projects and organizations? What challenges have you faced? What best practices do you recommend? What other significant differences do you see between traditional software vs ML/AI development?

Why AI should be a priority in your 2018 plans.

Artificial Intelligence has been at the top of many minds:

In the next 10 years, we will shift to a world that is AI-first. - Sundar Pichai, CEO, Google, 2016
AI is the next platform. All future applications, all future capabilities for all companies will be built on AI. - Marc Benioff, CEO, Salesforce, 2017
We need to be super careful with AI. Potentially more dangerous than nukes. - Elon Musk, CEO, Tesla, 2014

While a lot of this is either too optimistic or too pessimistic, it’s becoming clear that 2018 will be the ideal time for forward-thinking businesses to embrace AI.

Applied AI in 2018: Beyond the Buzzwords

I think of AI as “software that gets better with data.”

A confluence of trends is making 2018 the first year where AI will start to see mass adoption across diverse industries and in different sizes of businesses:

Trend #1 - Maturing Technology: It’s becoming easier and cheaper to build and maintain AI. For many applications, the challenge has shifted from technology to design, user adoption, privacy, and other go-to-market tasks. As Bloomberg Beta VC James Cham recently said, “[Product Management] is the biggest challenge in AI.”

Trend #2 - Increasing User Expectations: While most AI is not like Siri, Alexa, or Cortana, these voice-assistants are increasing user comfort and softening the ground for AI adoption. Even more so, for a lot of users, AI capabilities are becoming a must have, with the perfect example being the rapid growth of chatbot usage. 

Trend #3 - Higher Cost of Delay: AI advantage compounds over time. So, if you didn’t start two years ago, now is the next best time to start! This also makes the cost of delaying this decision exponentially worse. Your competition doesn’t need a head start. 

If you don’t have an AI strategy you are going to die in the world that’s coming. - Devin Wenig, CEO, eBay, 2017

It’s time to start applying AI in a practical and tangible way.

When considering this, what’s the next step?

Start with the most important question: Why?

Why is AI at the top of your mind? What could you achieve with AI?

The key is to focus on your business challenges and opportunities. The answers will help you decide how you prioritize, establish success criteria, and manage your investment.

While just “talking” about AI can give you a significant marketing benefit, responses to this “Why” question from senior executives typically fall within the following categories:

To capitalize on the AI Opportunity.

With AI, as with the industrial revolution, great fortunes will be made through innovation and lost through disruption. Even without the bold proclamations, it’s clear that AI presents a rare opportunity.

Broadly speaking, AI benefits any business in two ways: (1) Product/Service Innovation, and (2) Operational Excellence.

AI can make your existing product or service dramatically better. Recently, one of our customers demonstrated an early AI prototype to select prospects and the response was an emphatic “This is a game changer!”

AI is not one “thing”, but many different approaches. Depending on how you apply AI, it can help in many different situations: AI can help grow your business, increase profitability, improve customer experience, and build a strategic foundation for the next decade.

These possibilities are matched by the growing investment in AI-enabled companies.

“ [Excluding corporate R&D, VC investments in AI rose] from 160 deals in 2012 to 658 in 2016. Dollars invested rose 60% [to $5B+ USD].” - CB Insights, 2016

If you don’t have a clear idea of how AI applies to your business, consider exploring AI via training or prototyping. An “Introduction to Applied AI” session, typically attended by the entire executive team, is one of the most common starting points for our clients.

To win.

Competition is the leading reason why our customers approach us.

AI is a great way to create and maintain durable competitive differentiation. Since AI advantage compounds, the earlier one starts, the better. AI can also provide a brand and marketing lift that can drive revenue growth.

Only the paranoid survive. - Andy Grove, ex-CEO, Intel

Also, if you think your competitors and customers are not evaluating AI or investing in it, you may be unpleasantly surprised. Just recently, one of our customers’ competitor announced a big initiative to infuse their products with AI. By the time this competitor’s product launches, our customer will have had months of head start in a fiercely competitive market.

Customers are starting to demand AI-enabled products. There will be rewards for being first.

To attract and retain the best.

Sought-after talent join businesses that invest in the future.

On a recent AI project, over 50% of our client’s employees attended an optional kick-off meeting. Excitement, curiosity, desire to participate, and a desire to learn drove this unexpectedly high employee engagement.

Employees are keen to work on AI. Investment in AI will serve as an effective talent attraction, engagement, and retention tactic.

To build a core competency.

AI implementation will require some cultural and company-wide process changes, from marketing to sales to customer support. The sooner you start, the better you will be able to navigate.

For technology companies, AI development will become a long-term investment and a core competency, similar to databases, but much more powerful. AI is developed differently than typical software: it’s hypothesis-driven, probabilistic, difficult to debug, and experimental. Expect change, especially if you need to make long-term AI-related R&D investments.

It’s better to seek out change and adapt on your own terms before change is forced on you.

Answer to this “Why AI?” may vary for each business but… 

… investing in AI is becoming increasingly important.

AI can yield many benefits: from better products to higher profitability to improved employee morale.

Start in 2018. The more you delay, the farther behind you will start.