With just one AI product, it is possible to connect all the services needed by users in the medical and health sector.

Yesterday (September 5th), Alipay launched the "AI Health Butler" at the "2024 Inclusion·Bund Conference," covering more than 30 health services including finding doctors, reading reports, accompanying consultations, inquiring about medical insurance, and managing health. Behind these services are the first batch of over 20 medical intelligence entities, including local health and medical insurance departments, top-tier hospitals, key specialties, and authoritative medical experts.

At the same time, Alipay announced the opening of medical intelligence entity collaboration to the medical and general health industry, aiming to build an AI health service ecosystem.

"Two or three years ago, I told the team not to rush to chant the AI slogan, because most of it was still at the conceptual stage and couldn't bring about tangible changes," said Zhang Junjie, Vice President of Alipay and General Manager of the Digital Medical Health Division. However, now, with the advent of large model technology, the AI era has arrived, and Alipay is ready to fully commit to AI.

Advertisement

"The AI Health Butler will be the general entry point for our user services and the only thing we need to do in all our AI services to users, while the product itself also insists on being open to the ecosystem."

Before and after diagnosis management

Search for "AI Health Butler" on Alipay, enter, and you can see the continuously updated list of medical intelligence entities.

Among the more than 20 medical intelligence entities that have settled in, Dr. Zhang Tao, the chief physician of the Vascular Surgery Department of Peking University People's Hospital, is the real person behind the first doctor intelligence entity. At the launch event, Dr. Zhang Tao's team also shared their experience with the application of the doctor intelligence entity. The reason for being willing to be so "public" has to start with his experience in recent years of consultations.

As an authoritative expert in vascular surgery, most of Dr. Zhang Tao's energy is on the operating table, with his outpatient time being two half-days a week. Due to the infrequent consultations, he is almost always the one who goes the most overtime, from 8:30 in the morning until around 1:00 in the afternoon. What frustrates him the most is that most patients, within the limited consultation time, ask the most basic common sense questions. "Some patients can be explained in 5 minutes, while others may not understand even in half an hour, which is a significant time cost."After the diagnosis and treatment, the follow-up management of patients has been a pain point that has troubled Zhang Tao for many years. Taking patients with lower extremity deep vein thrombosis as an example, these patients need to undergo anticoagulant therapy after surgery to prevent the recurrence of thrombosis. Zhang Tao requires these patients to come back for a review of indicators every month to dynamically assess the risks and benefits of anticoagulant therapy. However, in reality, most patients will fall off his follow-up radar. They may be patients from other places, may not be able to get an appointment, or may not be able to obtain his feedback in time through online channels.

Therefore, when the Alipay team approached him to talk about the "Doctor Intelligent Body" project, the two sides almost hit it off immediately.

In the product design framework of the intelligent body, a mature Zhang Tao Doctor Intelligent Body can answer questions online 24 hours a day, achieving a high degree of consistency in accuracy and humanization. When users want to communicate further, they can directly jump to the doctor himself, and the intelligent body will organize the previous conversation into a file and push it to the doctor. At the post-diagnosis stage, the Zhang Tao Doctor Intelligent Body can also provide patients with report interpretation, medication reminders and other functions.

"In the traditional medical process, these tasks are mostly completed by the doctor's assistant team. But there will be two problems, one is that patients specifically want to find a doctor and do not trust the assistant. The second is that the level of the assistant team is limited, and the reply may not be timely." Zhang Tao told Jianwen Consulting that after several months of training and polishing, this intelligent body named after him has made rapid progress, from only being able to answer one or two sentences at the beginning, to now being able to draw inferences about other cases and cite classics, which is equivalent to the level of an excellent student in the medical assistant team.

Zhang Tao said that sometimes when he looks back at these conversations, he feels "hard to distinguish." His offline follow-up patients also continuously bring back feedback on the use, and everyone's unanimous evaluation is, "This thing is too magical, even the catchphrases are there, just like interacting with you in person." Most of the time, he will smile and add, "You still have to talk to me in person before prescribing medication and surgery."

"I am just the first to eat the crab, and there will definitely be more doctor intelligent bodies developed later." Zhang Tao said that in the past, information technology represented by internet hospitals mainly solved the connection problem between doctors and patients. To a certain extent, patients will only concentrate more on top doctors, increasing their burden. The doctor intelligent body solves the problem of efficiency, and can screen out patients who really need it for them, maximizing the value of famous doctors.

Can chat and handle affairs

Anyone who understands medical AI can see that the essence of the doctor intelligent body is a large medical model tailored for doctors.

Taking the Zhang Tao Doctor Intelligent Body as an example, its base is the Bailin large model independently developed by Ant Group. On this basis, the R&D team of Alipay has built three layers of data structure to continue training it. The first layer is the data of vascular surgery, including a large amount of literature, clinical guidelines, expert popular science, etc.; the second layer is the doctor's personal knowledge base, such as the case information provided by Dr. Zhang Tao, real consultation dialogues, etc.; the third layer is the doctor's personalized tuning, mainly from Dr. Zhang Tao's modification and annotation of the model-generated content.

Among these three layers of data, the third layer has the least amount of data, but it can best reflect the doctor's personal style and determine the similarity between the intelligent body and the real doctor. Zhao Yining, the product manager of the Digital Medical Health Division of Alipay, told Jianwen Consulting that when the first version of the Zhang Tao Doctor Intelligent Body was launched, only the first two layers of data were used, and the product form was a very ordinary Q&A robot. Later, with Dr. Zhang Tao's continuous personalized annotation of the model's answers, the intelligent body began to slowly have character traits, using some modal particles, and comforting patients emotionally.In the recently launched version, users can not only communicate on a textual level but also convert the intelligent agent's reply text into Dr. Zhang Tao's voice, akin to a celebrity providing navigation announcements, thereby enhancing the doctor's personal characteristics. The next step will be to add a "phone call" function to the doctor's intelligent agent, allowing real-time conversations with users, just as if they were speaking directly with Dr. Zhang Tao himself, which is convenient for the elderly population.

In fact, from a cost-benefit perspective, large model products based on personal customization are not mainstream. The existing digital human avatars on the market are also mostly based on one-way output for science popularization broadcasts, with very few interactive features like this.

Alipay is willing to undertake this initiative for two main reasons. First, it has the support of multimodal medical large models and digital human capabilities, which allow for complex interactions while keeping costs relatively manageable. Second, in the medical context, patients are accustomed to seeking help from a real doctor rather than search engines or other virtual robots, which makes a significant difference in user experience.

Another, more hidden reason may require us to look beyond the doctor's intelligent agent product. Over the past decade, within the medical service ecosystem built by Alipay, there are numerous functions beyond consultation, such as registration, accompaniment, medication purchase, and medical insurance payment. Although they are all integrated into one app, each service has its own entry point. The biggest difference between an intelligent agent and a Q&A robot is that the former can not only chat but also activate these services.

"We hope that users can interact with reality through the intelligent agent, whether you want the doctor to review a report or register for a follow-up visit and purchase medication, the intelligent agent will handle it for you, instead of just telling you what to do," said Zhao Yining. The real diagnostic scenario is complex, with different services each having their own entry points, and Alipay aims to handle everything through a single online entry point.

Furthermore, according to Alipay, several intelligent agents of senior medical experts will be launched soon, covering departments such as breast surgery, sleep medicine, sports medicine, and traditional Chinese medicine.

Openness and integration are key.

Among the first batch of more than 20 medical intelligent agents to join Alipay, in addition to the main body of doctors, there are also local health and medical insurance departments, tertiary hospitals, and key specialties. For example, "An Zhen Er" is the first medical service intelligent agent jointly built by Alipay and the Zhejiang Health Commission, which has been implemented in more than 1,000 medical institutions in Zhejiang and has served nearly 5 million people.

In the view of Zhang Junjie, Vice President of Alipay and General Manager of the Digital Medical Health Division, medical intelligent agents at different levels have different functional positions and varying implementation difficulties. For instance, intelligent agents co-built with local health and medical insurance departments and tertiary hospitals mainly address the standardization of business processes, which is challenging at the beginning but easier to scale once a model is established. On the other hand, intelligent agents focused on specialties and individual doctors require a higher level of professional expertise and demand deeper cooperation for refinement.

Regarding the scale of intelligent agent cooperation, Zhang Junjie predicts that the next two years will be a critical phase for expanding from specific points to a broader scope.At the press conference, Alipay announced the opening of a professional intelligent agent collaboration ecosystem, supporting free access to medical large models, and opening up supporting computing power and data security solutions. At the same time, it will also invest more than 100 million yuan in special fund resources to work with medical institutions to develop innovative health intelligent agent solutions.

"We hope that the AI Health Butler can play three good roles, the first is a health consultant, the second is a medical companion, and the third is a doctor's assistant. Therefore, in these three directions, as long as it can provide assistance, it is a partner we will focus on and prioritize cooperation with," said Zhang Junjie.

After the launch of the AI Health Butler, the business landscape of Alipay's medical health has also become clearer. Previously, Alipay proposed three main business lines: convenient payment tools, medical treatment and medication assistants, and medical health butlers. Now, after the medical health butler has been fully AI-ized, it has integrated the former two in the business flow, forming a more closely connected service matrix.

"In fact, the concept of the AI Health Butler is not new at all, but who can do it down-to-earth and long-term, and do a good job of the user experience, this is an opportunity, and we are confident and patient to do this well," said Zhang Junjie.

Zhang Junjie said that Alipay's advantage lies in connecting more than 600 million medical insurance users, full-process services of more than 3,600 hospitals, and leading enterprises in the general health field. What they need to do now is to connect these services through AI. Although this is a heavy task, it is also their barrier.

"I have always believed that the medical field needs a platform to do what the platform should do. Simple connections are definitely not what we want. What we pursue must be cross-disciplinary and further integrated, only in this way can we possibly create new value."