Source: Quantum Bit

Image source: Generated by Wujie AI

This year's Double 11, in addition to buyers, sellers, and platforms, there is a fourth participant:

AI assistant.

On the Taobao and Tmall platforms, the number of times merchants have used AI during the Double 11 preparation and sales period has exceeded 1.5 billion.

On the buyer side, the cumulative user experience of the AI assistant Taobao Wenwen has also exceeded 10 million, with a daily average of over 8 questions from highly active users.

The most exaggerated data is "a single user asked more than 4000 questions", does this person completely rely on AI to decide what to buy without doing any research?

This time, various AI applications have blossomed on Taobao and Tmall.

Not sure what to ask AI, which specific product to choose, how to place an order after making a selection, and which activities are cost-effective? You can ask AI for all of these.

Furthermore, sellers can quickly open new stores with AI, list a large number of products with AI, and even run advertisements with AI.

……

The explosion of large models has been ongoing for nearly a year, continuously transforming people's work and online entertainment methods.

And penetrating into e-commerce shopping, it can be considered as connecting to a broader range of offline life such as clothing, food, housing, and transportation.

New AI Shopping Experience

The AI assistant Taobao Wenwen has been opened to all Taobao users, you can directly access it by entering "Taobao Wenwen" in the search box.

At first glance, it looks similar to other chatbot products, with a chat box as the main interface.

But in actual experience, it not only understands products, but also understands you.

If you need to meet the minimum amount for a discount coupon, you can ask AI to recommend some snacks to add to your order.

Without adding additional requirements when asking questions, it can also recommend products that match your preferences based on historical orders, shopping carts, and other data.

For each recommended product, AI will summarize the product's characteristics, suitable target audience or occasions, and the reasons for the recommendation.

By clicking on the product card, you can directly select the specifications and flavors to add to your shopping cart.

In addition to direct questioning, there are some built-in function templates in Taobao Wenwen.

"Help me choose products" is specifically designed for those with decision-making difficulties, clearly comparing the advantages and disadvantages of two products.

"Wedding planner" and "Travel planner", "Senior shopping guide" are all optimized for their respective scenarios, presenting overall plans and recommending products for each step in one go.

"Life assistant" and "Foodie" can solve daily problems and also help you purchase all the necessary materials with one click.

Finally, "Soul writer" is prepared for users who like to share their shopping experiences.

Here's a detailed introduction to the "Help me choose products" comparison gameplay: directly select two products from browsing history or shopping cart that you are unsure about.

Even for different categories such as laundry detergent and laundry pods, AI can help you analyze the similarities and differences.

Of course, if you want to compare clothes and figure out how to match them, Taobao Wenwen can't help you for now.

At this point, you need to use another function "Taobao Try On", which can also be found by searching directly.

Click on the item you like to change outfits, you can mix and match tops and bottoms or choose a set, it's like a real-life version of Love Nikki.

For those involving matching with different hairstyles, face shapes, body types, and leg shapes, you can also upload a full-body photo of yourself and set your height and weight to replace the AI model, allowing you to try on millions of outfits without leaving home or spending money.

You can click the compare button to repeatedly compare with the state when you took the photo, which is also a very practical gameplay.

In addition to the above, Taobao and Tmall have also prepared "My Cute Pet", which uses AIGC technology to create a digital avatar for your pet; "Extreme Home", where you can upload room photos and design and decorate online, and many more features.

All can be accessed directly by searching in the search bar. Due to limited space, I'll leave it to interested readers to experience it themselves.

Merchants also have a dedicated "Taobao Merchant Intelligent Operation Tool", from uploading the first product photo, naming the store, generating a logo, automatically identifying product attributes based solely on the image, listing, to store decoration, all in one go, saving a lot of manual operations.

, duration 01:30

In addition, there is also a function to freely expand the size of product materials, adapting to various display occasions, greatly reducing material processing costs and avoiding rework.

Finally, from the review data, this year's Double 11 saw the highest participation from new brands, new merchants, and small and medium-sized merchants.

As usual, unfamiliarity with platform features and lack of proficiency in operations can cause a lot of trouble, but this year, with the introduction of new technology, the threshold for operating a store has been significantly lowered.

The Taobao Merchant Intelligent Operation Tool has been online for less than half a month and has provided over 100,000 AI product launch services for women's clothing and apparel merchants, reducing the product launch time by 25% compared to traditional methods.

Co-piloted by the centralized e-commerce operation Copilot, developed based on large language model technology, after the start of internal testing, it has provided business knowledge Q&A, operation tool calls, and copy generation for tens of thousands of merchants, and has already improved business efficiency for merchants over 500,000 times.

The First Large-Scale Application of AI Technology in the E-commerce Field

If we count, ChatGPT has been released for almost a year, and the rise of AIGC text-to-image came even earlier.

During this period, every technological breakthrough has sparked a wave of entrepreneurship or the transformation of existing businesses. Until the Double 11 period this year, AI in the e-commerce field finally achieved large-scale application, which is of extraordinary significance:

It is not only used for internal enterprise processes but also for end users. Moreover, it is a terminal user that faces two completely different characteristics and needs of buyers and sellers.

This involves the integration and innovation of two major technological routes, large models, and AIGC text-to-image, which can observe how AI crosses the gap from technology to application and can also provide inspiration for more industries.

Let's talk about the earlier rise of AIGC text-to-image. In August 2022, Stable Diffusion was open-sourced, with low computing power requirements, marking the beginning of the commercialization of AIGC.

However, Stable Diffusion's weaknesses were quickly exposed in practice:

First, there was the issue of uncontrollable image generation, especially in the e-commerce field where the accuracy of presenting the subject of the product is highly demanded.

Secondly, it was complex to use, and the complex prompt word skills were difficult to master, and there were many parameters that needed adjustment, increasing the learning cost.

The turning point quickly appeared. In January, ControlNet emerged, cleverly solving the controllability issue to some extent by adding auxiliary conditions to guide generation, and was hailed as "changing the rules of the game".

At this point, experts could already use the combination of Stable Diffusion and ControlNet to specify the details of the generated images, such as character posture and overall layout, effectively raising the upper limit of AIGC text-to-image.

However, for the many ordinary users that Taobao Tmall serves, the understanding and usage cost of these guiding methods remained high.

In August 2023, a new technology IP-Adapter appeared, once again lowering the threshold for stable on-demand image generation.

IP-Adapter is equivalent to opening up a new method of "using images as prompts" and can also be considered as a guiding condition in ControlNet.

As a result, e-commerce sellers can use existing product photos in the text-to-image process, without the need to learn and master prompt word skills or other professional graphic knowledge, further lowering the lower limit of AIGC.

While the basic practicality issues have been resolved, deeper-level needs have emerged.

Stable Diffusion is a pre-trained model, and its understanding of the world mainly comes from the data in the training phase.

And the e-commerce field is rapidly changing, with a large number of new products being launched every day, requiring AI to continuously update and learn.

At this point, the fast fine-tuning technology LoRA from the large language model comes into play, serving as a "knowledge/concept injection" method in the AIGC text-to-image field.

Specifically, during fine-tuning, LoRA freezes most of the model's weights and only updates a small part. The updated weights can also be separated individually, each only a few tens to hundreds of megabytes in size.

In practice at Taobao Tmall, LoRA has essentially created digital avatars for products and models, allowing merchants to accumulate digital assets, which can further generate a more diverse range of product or model images.

After adding LoRA to the process, there were issues with generating controllable but poor-quality images. How to balance the generation of models between realism and beauty, and how to accurately restore product details based on models generated by LoRA, are application challenges that need to be further addressed.

In addition, Taobao Tmall is also exploring a new knowledge injection technology that can apply product concepts to image generation without training, directly providing product images in the inference phase, greatly reducing the application cost of knowledge injection technology and improving real-time performance.

For AIGC to be widely applied, there are so many enhancements and transformations around the image generation model Stable Diffusion itself.

But in actual workflow, a lot of work needs to be done to significantly reduce the complexity of image generation operations.

For example, in response to the e-commerce field's requirement for accurate presentation of the product subject, the SAM segmentation model was added, which separates the product subject first and then adapts and integrates it with the AI-generated environmental background.

This ensures the accurate presentation of the product subject, avoids "mismatched goods," and seamlessly integrates the product subject with the background, making the light and shadow appear flawless and giving a sense of placement.

Next, it is necessary to combine local detail repair, super-resolution, and adding filters to improve image quality and achieve commercial-level standards.

In this way, the application of AIGC, and the journey of large models towards large-scale application in the e-commerce field, is equally difficult.

Especially for Taobao Tmall, it is not just about a professional model, but a general model with strong professional capabilities, to analyze the long-tail natural language problems of Taobao users and understand more precise user intent.

To achieve this, it is necessary to first add e-commerce professional knowledge to the general model and train it with data accumulated in the e-commerce industry over many years.

But at this point, the general large model is still continuing to write text based on the input, for example, when a user asks a question, the model is likely to supplement multiple similar questions according to the format, rather than answer the question.

In the industry, the method to solve this problem is to align AI with human preferences. SFT (instruction fine-tuning) allows the large model to learn how to fulfill user instructions, and RLHF (human feedback reinforcement learning) allows the large model to learn what kind of response satisfies human preferences. In practice at Taobao Tmall, the product can continue to iterate based on user feedback preferences after it goes online.

The next challenge is the "illusion problem" in the large model's responses, which the technical team at Taobao Tmall is addressing from both inside and outside the model.

Internally, that is, during model training, a large amount of e-commerce industry data is introduced.

Externally, through the RAG (retrieval-augmented generation) technology, different knowledge bases are called for different questions to obtain real-time updated product information.

Furthermore, in response to the need for real-time access to external tools, the technical team at Taobao Tmall has utilized Tool learning technology to optimize the large model's understanding, selection, and invocation of tools, and provide interpretable tool invocation paths, making the responses more accurate and diverse in form.

For example, in a typical user query on Taobao's question and answer platform, not only the capabilities of the large model are invoked, but also the product recommendation capabilities of Taobao and the video recommendation capabilities of the content community. In more complex scenarios such as travel planning, the flight and hotel booking capabilities of the partner Fliggy may also be invoked.

Ultimately, a series of large model products represented by Taobao's question and answer platform have intent recognition, task planning, memory, and the ability to use external tools on the basis of large models, and are evolving towards the direction of an Agent (intelligent entity), which is the next competitive stage in the large model industry.

How did AI perform in the Double 11 shopping festival?

In the past, every Double 11 was a major test for technology and engineering. This year, with the addition of AI, it added a layer of the taste of the first real-world test.

Now, does this wave of AI achievements from the Taobao Tmall Group also represent some clear new trends?

First, AI technological innovation has expanded the boundaries of e-commerce and increased possibilities.

Shopping has changed from needing to know exactly what to buy and then searching for it, to being able to ask AI for anything you need.

Even if you didn't know what to buy before or didn't think your needs could be met by purchasing products, AI may recommend something for you.

Second, AI technological innovation has improved the working methods and production efficiency of merchants.

Similar to buyers but different, AI recommends backend functions and operational tools to sellers.

In the past, merchants using the Taobao Tmall merchant backend's Qianniu Workbench had a high learning curve, with too many functions, many of which they didn't know where to find, or even that a certain function existed.

With AI, the process of running a store has become one where any need can be addressed by AI, allowing AI to locate the appropriate function or answer the merchant's questions using RAG technology.

Finally, although Double 11 has passed, a large amount of interaction with AI has left valuable experience and data.

The Taobao Tmall Group already has complete e-commerce industry data and experience, and with its scalable AI products, it generated a large amount of user feedback data during Double 11, which can be reinvested into algorithm improvements, ultimately creating a snowball effect.

The OpenAI Developer Day marks the increasing momentum of the AI industry, and YC founder Paul Graham has suggested that to avoid being eliminated in competition, one needs to:

• Rely not only on AI but also on deep domain knowledge in a specific field
• Establish very close relationships with end users

And these two points happen to be the strengths of the Taobao Tmall Group.

In the AIGC industry, there is a saying, "AI progresses a day, equivalent to a year in the human world." We look forward to what new shopping experiences AI will bring us in next year's 618 and Double 11.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。