Carrying Thousands of People Activities through Real-time Data Push|Corporate Practice

Carrying Thousands of People Activities through Real-time Data Push|Corporate Practice

At this year’s annual meeting of Tencent’s mutual entertainment market system, the tgideas team produced a WeChat applet with offline services as its mainstay. It participated in many sessions of the staff meeting and dinner. The applied technologies include: company smart gateway + iBeacon Judging the identity, the applet pulls up navigation, applet cloud functions, cloud database, cloud storage, multi-scene real-time database, applet payment ability, embedded H5 game, applet red envelope ability.

background

This small program is an auxiliary tool and interactive tool for the TIEM market system annual meeting. The annual meeting has 1200+ employees and dozens of external guests. The annual meeting is divided into the afternoon staff meeting, red carpet and dinner performances. The specific function points of the mini program and the corresponding core technologies are as follows:

  1. The smart gateway determines whether it is an employee, and iBeacon determines whether it is an on-site guest.
  2. The real-time database updates the progress of the conference + red carpet + dinner program.
  3. In the live debate session, the real-time database displays the votes and displays them on the side screen of the stage in real time (Like Qihua said).
  4. The real-time database updates the red carpet process (the process in which each team in the department queues up for admission).
  5. The wonderful scene is stored in the cloud in a waterfall flow mode, providing employees with the functions of uploading photos and previewing photos.
  6. In the dinner program, free support and paid rewards can be provided for the programs performed by the employees, and they can be presented in the mini program and the stage side screen through the real-time database .
  7. The results of each web lottery draw on the big stage screen are synchronized to the applet cloud database via HTTP API .
  8. During the dinner, the whole H5 interactive mini game, each player's game score is summarized in the total score of the table he selected through HTTP API , and the table is the unit to compete for the first place, and the top three tables are displayed on the stage screen through the real-time database capability. Number and total score.
  9. Everyone who has experienced the game will be allocated a cash red envelope with the ability of the applet red envelope after returning to the game lobby of the applet .

Cloud development

This is my first contact with cloud development. I have to start from scratch and complete the task within one month. Under the pressure of heavy tasks and tight time, I also worry about the ability of cloud development. It turns out that the out-of-the-box use of small programs and cloud development, the integration of native WeChat capabilities, the built-in authentication of cloud functions, and the ability to obtain openid are very powerful for front-end development such as me without background development experience. Good program choice.

Real-time database

What I want to share the most in this case is the ability of Tencent Cloud's joint small program to launch in the second half of the year- real-time data push

The real-time push capability provided by the WeChat applet is very simple. There is only one API, which is watch[1] Remember this keyword, it is synonymous with real-time data push. Official self-developed capability. It can be used out of the box, no need to manage long links, no need Write server-side code without occupying the number of database connections (thanks to Zhou Zijie [2] and Deng Kunli [3] for the answers). In short, it is an official gift. From a chat scene included in the official cloud development demo, it can be seen that the real-time data push capability is inherently friendly to chat rooms, chat modules, etc. that require instant messaging functions. It also fits well with the "instant feedback after reward" function of our annual meeting mini program and the game lobby function in the mini program. Next, let’s talk about how we apply real-time data push capabilities. 

watch application-full rhythm control of the annual meeting

When I relaunch the applet, I set up a watch, monitored adminConfig, and wrote the monitored data changes into the data of the current page, so that the interface state can be changed.

First of all, you can feel that the small program management terminal on the left will have corresponding changes in the small program terminal of all users on the right as long as there is an operation:

 1//Business logic is too complicated, write pseudo code here
 2//app.js
 3App({
 4 loadConfig:function(){
 5//You can think of it as a setInterval, set an ID to it, it is convenient to close
 6 this.globalData.adminWatch = db.collection("adminConfig").watch({
 7 onChange: function(res) {
 8 let {
 9 _id,
10...adminConfig
11} = res.docs[0];
12 console.log('Configuration changes:', adminConfig)
13 const _page = getCurrentPages();
14 if (_page) {
15 _page[_page.length-1].setData({//This is very important, the monitored changes drive the current interface changes
16 adminConfig
17 })
18}
19 },
20 onShow:function(){
21 this.loadConfig()//Enable monitoring every time the applet is invoked
22  },
23 onHide:function(){
24 this.globalData.adminWatch.close()//Close the monitoring every time the applet is hidden
25}
26})
27
28//For a specific page, such as game.xml, you can rely on the above adminConfig for status judgment.
29<view wx:if="{{adminConfig.dinnerGame}}">The game officially starts</view>
30<view wx:else>Play the game</view>

前文说过,我们的小程序管理端页面,其实都是在对一个名为 adminConfig 的 colletion 做修改。而我们在打开小程序的第一时间就开启监听此 adminConfig,当它有什么变化,则会驱动我们的界面相应变化。而我们也需要设计小程序的每一个需要被控制的页面,结构与逻辑都想办法依赖这个 adminConfig,不然没办法驱动此页面的界面变化。比如下图,我们在管理端点击“小游戏入口开启”后,大家的小程序界面瞬间多了一个 banner 入口;点击“游戏开始”后,大家的小程序界面则会瞬间从“开始试玩”变成“点击进入”。

我们就是通过实时数据推送,做到在每一个员工/嘉宾的小程序界面上,实时更改他们的界面,改变员工大会晚宴节目进程。

在 watch 面前,连 setData 都有可能成为性能瓶颈

打赏页面的逻辑是,用户在打赏的时候,我们就在 colletion 为 rewards 的集合里 add 一条记录。然后我们的实时数据推送 watch 将监听整个 rewards 的数据改变,watch 的 onChange 事件给出了什么数据,我们就把这些数据洗一遍,然后直接 setData 到我们的界面上。

看起来没问题,自己测试的时候也没问题,每次送上小爱心,界面的打赏记录也会瞬间反馈我的打赏行为。我也觉得这一切都如此合理。

其实这里存在了一个很隐蔽的问题,如果不是我们利用年会演习的机会测试了一次,如果不是我们的总PM心细如发体验到了此问题,估计到了现场将会是一个灾难

存在的危机:当有很多人同时在打赏的时候,watch 的 onChange 几乎是无微(每条记录)不至(推送到达)的,在每一次 onChange 都会反馈到小程序端,也就是每次都会触发 setData 去驱动界面渲染一次。watch 并不是我们想象的,会缓存一波数据改变再推送过来,它的反馈是如此直接暴力,用户 add 一条,它就推送一条,数据落盘 DB 到数据从服务端推送出去,这里仅仅是 5-10ms,也就是说除非在这 5-10ms 内有多个用户同时 add 数据,才会出现返回的记录 length 是 2 的情况。试想如果每秒都有 10 个记录反馈,那 setData 就得操作 10 次。而 setData 是一个异步行为,也就意味着它的执行是需要时间的。我们如果操作唤起“立即打赏”面板(其实也是一次 setData 操作),则会面临,我们需要在 1 秒 10 次的 setData 间隙中,插入这次的唤起界面的操作,但是试想我们怎么能有设备快?setData 一旦空闲,立刻就被 onChange 事件的反馈自动执行并占用了。而我们的唤起界面的操作就被阻挡在外了,连排队的资格都没有(setData 没有排队的设定)。

如果在现场大家想打赏却唤不起打赏面板,那将是很尴尬的事情。如果涉及到收入什么的(比如此处涉及到了微信支付),那就是运营事故了~~

问题已经发现并抛出来了,解决起来就简单了。我们的解决方式是:

1onChange推送过来的数据堆在一个arr变量里:
2onChange:function(res){
3 _this.arr.push(res.doc)
4}
5然后setInterval一个函数,每秒执行一次:
6  {
7    把arr的值setData
8    然后再清空arr
9  }

Is it more intuitive for me to describe the code in this natural language? This makes it difficult for the watch to occupy setData when the setData that invokes the interface is clicked to execute.

One more note here is that watch has a capacity limitation, which can only monitor 5000 records of a collection. For example, the reward database above is designed to add a record for every reward. If there are more than 5000 items, the watch event will report an error.

So we must pay attention to the design of the database to avoid the situation where the watch collection exceeds 5000 records! watch takes another channel, does not occupy the number of simultaneous connections identified in the package, but supports up to 1W of monitoring by default . For example, the design in adminConfig I mentioned before has only one record, even if it is a global watch, there is no pressure.

watch application-game lobby

The game lobby is probably the most expensive part of my entire Mini Program. Here is the first interface:

When I first entered this interface, I didn’t take a seat. The table number of the table where I was dining corresponds to the table number here. Select the table number to be seated (the host will read it out); if you find that your table number is wrong after seating, you can click directly You can enter the seat again with the correct table number, or you can click to leave first, and then choose the correct table number to seat. This involves two cloud function calls. In a weak network environment, it may cause a delay of 1 second. If someone chooses the same table number as you during the delay, there are already 11 people at this table (there is a limit of 12 people per table) , It will cause data errors. Therefore, the query conditions and update methods are very particular. After 3 and a half revisions, I optimized the experience to the point where everyone can barely accept it.

The first time: The design of the watch itself is unreasonable, just to run through the entire interaction, to reach the basic goal of joining, leaving, and starting directly, but not only the performance is not good, it takes 1.5+ seconds to click into the interface to complete the display, and the front-end performance Inconsistency also appeared, mainly reflected in the avatar already on the table, but the "Start Directly" and "Leave" buttons were displayed after a delay of 0.5 seconds.

2.time: Optimize the table structure. Each table is used as a fixed table record. Each player is just an object in a member field array in this record. However, this optimization appears. When you frequently click "Join" other tables , There will be multiple BUGs of your own avatar. That is, one user can occupy multiple tables, which is obviously unreasonable. Moreover, the logic of table changing involves 4 cloud function call calculations, and the cloud function calculation time is about 800ms.

The third time: After a deeper understanding of the cloud database, the optimization of the cloud function has changed from 4 calls to 2 times of table changing. Previously, it was to query whether the target table is full, and then insert data; after optimization, it is directly inserted. If it cannot be inserted, the status code of status='0' is returned to notify the front end. Avoid the bug of multiple avatars when changing tables, and avoid data abnormalities caused by 11 people at a table and 2 people grabbing the table at the same time. The computing time of cloud functions is also shortened to less than 400ms. When jumping tables, first do the operation of adding personal information to the target table, and then do the operation of deleting personal information of the current table, so that the delete time does not need to wait.

Fourth time: Optimize the interaction, add a showLoading when you click to join, to avoid static waiting, but also to avoid rendering bugs caused by repeated table jumps under a weak network.

Later, Lin Chao from the WeChat Mini Program Group added that the screenshot above is a console screenshot of the cloud function debugging interface. Because it is a development tool, the time-consuming is much greater than the real time-consuming on a physical machine. We can refer to the data to achieve optimization purposes.

Summarize the key optimization points of the game lobby:

  1. The structure design of the data table, it is best to use the unit that can estimate the upper limit as the record itself, such as the number of tables;
  2. The data operation to join the table is update. The increase of member is self-incremented by the database API _.inc, and multiple users write at the same time. For the database, the field is self-incremented, and there will be no case that latecomers overwrite the former;
  3. If it is a jump table, first run the join table logic, and then leave the current table logic;
  4. It must be designed to prevent multiple clicks;
  5. Partition (1-30 tables, 31-60 tables) rendering, partition watch, the details are in place, you don't need to watch all the tables (1-120 tables) all the time;
  6. Fewer queries, more detailed query conditions;
  7. Adding showLoading with mask in appropriate links can avoid mutual interference of user behavior.

For this kind of multi-person participation, they will have a mutual relationship with each other, and the immediacy requirements are relatively high. For small program scenarios that also need to ensure data consistency, the follow-up suggestion is to give yourself twice the time to deal with it, so as to ensure the final landing. , Bring a good experience. Optimizing from one direction blindly may not be a good way. For example, the operating bottleneck of cloud functions is a callback time of at least 150+ms, so no matter how you optimize the experience, it will not exceed this category; and if you add appropriate loading animations to block other operations, you will virtually reduce users. The awkward waiting also improves the stability of the interaction.

data

In the 3-hour annual meeting, the applet carried 86175 PV and 1577 UV. In such a short period of time, with such a large amount of visits, cloud development completed this task very easily.

summary

The mini program of TIEM’s annual meeting, from scratch, from design to R&D to testing, took about one month. When our team was in a panic until the end of the annual meeting, we successfully completed this slightly beyond us. After estimating the scope of the task, everyone let go of their hanging hearts. After all, when we resumed the market, we discovered that we still had many failures, such as the weak network environment test for offline activities . Therefore, some colleagues cannot start H5 games in the game lobby.

The affirmation given by our leaders and colleagues and some pertinent suggestions made are the harvest of our entire team.

And my own gains are great. I used to feel that cloud development is a very distant job. After all, we have always had specific positions doing this part of the work, and I only need to use the downstream interface he provides. After this project, I not only learned cloud development and cloud functions in various ways, but also deeply explored the ability of small programs to push real-time data. As a front-end er, I can also combine my understanding and insight into the interactive experience to polish the overall interaction and experience of the applet, make up for the closed loop of the initial design of the interaction, and make the entire product appear more rounded and more in line with the operating habits of natural people. . At the same time, I have a deeper understanding of the support of offline activities, and I have tasted the hardships of the backstage staff.

TGideas community TGIDEAS team

Reference

[1]watch: https://developers.weixin.qq.com/miniprogram/dev/wxcloud/reference-sdk-api/database/collection/Collection.watch.html

[2]Zhou Zijie: https://new.qq.com/omn/20190815/20190815A0NTXB00

[3] Deng Kunli: https://cloud.tencent.com/developer/salon/salon-1132/agenda-10004

Cloud Development (CloudBase) is a cloud-integrated product solution. It adopts a serverless architecture, avoids operation and maintenance such as environment construction, supports one cloud and multiple terminals, and helps to quickly build small programs, web applications, and mobile applications.

Technical documentation: https://www.cloudbase.net/

WeChat search: Tencent cloud development, get the latest progress of the project

Reference: https://cloud.tencent.com/developer/article/1607521 Carrying Thousands of People Activities Through Real-time Data Push|Enterprise Practice-Cloud + Community-Tencent Cloud