• Services
  • Blog
  • Contact

Make New Programs Succeed​.

Be Honest: Are You Building a Commercial MVP, or a Prototype?

9/23/2020

3 Comments

 
Picture
In many industries, sales is used to ‘selling’ the vision of the product in advance of it being built, and customers assume vapourware by default. No one bats an eye because we’re accustomed to the idea that engineering will always be able to fulfill whatever we’re selling, given enough time and money.

New technologies like machine learning and blockchain offer a world of possibilities, but many of these possibilities may not actually be able to be implemented in practice, even with a huge budget. It's easy to promise "The product will automatically predict X with high accuracy." where X could be anything from detecting a security breach to predicting stock prices to finding the perfect outfit for you wear. But even if the prototype is already 70% accurate, it may never get to 80%, or whatever you need it to be to be commercially viable.
Don't oversell the vision, be honest. Be honest with customers, and be honest within the company walls too. Imagine slightly implying or exaggerating results to your manager, who then implies or exaggerates a little further to their manager, etc. and pretty soon the board is hearing that the product is 110% accurate and can slice bread too!

​
Be Absolutely Clear: Is This a Prototype or a Commercial Product? 

When starting a traditional project with new technology, it’s common to start with a prototype. Although best practice is to "throw away" that prototype and start fresh on the commercial development, management often pushes teams to continue work directly on the prototype to turn it into a commercial product, skipping the throw away step. The result is a shaky development foundation that can result in code debt and bugs for the lifetime of the product. Far from ideal, but it can be liveable in a business sense.

But let's take the example of an AI/ML project. The mechanics of delivering such a program are very similar to traditional projects on the face of it, so it is tempting to think of the implementation as being the easy part. But because of the non-deterministic nature of AI/ML, all the pitfalls of ordinary software development programs - misaligned objectives, underestimation, lack of process, skimping on QA, ignoring risks, etc. - are amplified x10. To deliver successfully, you need to ratchet up the program diligence (see Delivering an AI Program: What Could Possibli Go Wrong?)

Skipping the throwaway step is no longer possible. When a data scientist is prototyping a model, their focus is truly to validate that the product could ever have a chance of being delivered, and if not, what other related things it could do instead to deliver value. Prototyping AI involves many shortcuts - working on a "clean" data set (artificially clean, biased, small sample), where the availability and timing of the data may not be an issue. The work is done on a Jupyter notebook or other simulated environment that is vastly different from a real-world environment, and using models and techniques that could never be used in a commercial context. The business problem itself may be fluid, as the data scientist may realize that the algorithm can't solve X but can solve Y instead.


When the prototype has met its objectives, a new project should be started from scratch for the commercial development. In commercial development, the team's work is focused on entirely different things such as:
  • how to implement the model at scale
  • on large sets of data
  • on imperfect real-world data that may not always be available in time or at all
  • on data that may require considerable ETL
  • and ensuring the model's predictive performance is not only good when you develop it but continues to deliver accurate results even as time goes on in the field and data may change (known as drift)

So, is the project a prototype or a commercial endeavour? Alignment from the team to upper management is critical. The answer can't be both.

Related reading:
  • How to Deliver an AI/ML Project (What Could Possibli Go Wrong?)
  • How to Build Your Own Data Science Team for AI/ML Projects
  • AI Product Manager? What Your R&D Team Needs To Know

3 Comments
noahawilliam link
5/24/2021 08:23:39 am


Hi, every day I visit a number of blog sites but I'm very happy to read your post Thanks for sharing the informative blog post about <a href="https://vh1roofing.com/"> roofer in tulsa
</a>

Reply
website link
10/7/2021 01:01:36 pm

Mainly via making an intervention from the maximum flimsy, transparent and disposable material i ought to get right of entry to, might make humans each re-evaluate the gallery and its raisin.

Reply
galuapluservices-2 link
12/1/2021 06:27:45 am

Excellent article! Your post is essential today. Thanks for sharing, by the way.

Reply

Your comment will be posted after it is approved.


Leave a Reply.

    Categories

    All
    Artificial Intelligence
    Business Development
    Customer Intelligence
    Data Privacy
    Data Protection
    Demand Generation
    Growth Hacking
    Industry Analysis
    Leadership
    Market Opportunities
    Product Management
    Product Market Fit
    Program Delivery
    Project Management
    SaaS
    Strategy

Proudly powered by Weebly
  • Services
  • Blog
  • Contact