Making sure we share the slogan of “letting every single student pass” within the team even though the team has many members
Q：Please, tell up briefly about “the Coaching Course for University Entrance Exam Preparation”.
Mitsuoka: At this course, we provide each G10-12 student with a study plan suitable for his/her desired university, and support until he/she passes the entrance exam. When users, who have registered our free trial, reach the top page of the course, there are some videos waiting for them. We call those videos “the guidance”. The students will watch the guidance and understand “what the Coaching Course for University Entrance Exam is, and the course has the chatting function”, and start exchanging messages with our registration operator in the chat room. Then, a university student will be assigned as a coach based on the correspondence in the chat room, and the coach starts supporting the study of the user.
Q: I think the service is supported by a large number of members.
Ikeda: Yes, there are quite a lot of people who are related to the product. There are business-related people as well as people who deal with content production, and engineers who develop the product. There are also around 230 university students who are working as our coaches, and external outsourcing partners who help our coaching service.
Q: That is quite a big family. Are there any philosophies and/or policies everyone shares?
Ikeda: Yes, we have this slogan of “letting every single student pass” here in the Coaching Course for University Entrance Exam Preparation. First of all, we aim at letting each one of the users pass his/her desired university. Without altering this initial goal, we make policies on what we should do to realize that. There are also additional key phrases we often hear in the meeting, such as “what is the best for the user?” and “user-first” etc. 
Greatly improved the first charging rate by changing the assigning rules and coordinating the guidance at registration
Q: What was the issue the course had regarding the first charging rate?
Nakanishi: The main target of the Coaching Course for University Entrance Exam Preparation is G12 and graduate students. We start accepting those students in March and finish accepting them by July. On the other hand, our target between December and January is G10-11 students. However, we had this issue of low first charging rate of G10-11 students. For G12 and graduate students, the entrance exam is just around the corner and their motivation is high, so both of the number of starting users and the first charging rate are naturally relatively high. Contrarily, the first charging rate of G10-11 students was low, and we all wanted to improve that.
Q: What sort of measurements did you take for that?
Nakanishi: There were many measurements. However, I think two of them were particularly significant. First, we coordinated our guidance so that new users would experience various things of the course during the 14-day free trial. Second, we devised how to assign “a coach” who supports each student with communication. My role was to give the team feedback through monitoring regarding if these measurements had been effective or not.
Mitsuoka: At the Coaching Course for University Entrance Exam Preparation, we have a 14 day free trial period. Therefore, it is very important for us to let users feel the value of this service by communicating with the coach within the first 14 days.
Our course can be used with a smartphone app as well as on the website, and we started by estimating that how the user is accessing the course may be the key. If a user is using the app, he/she will receive automatic notice of a message from the coach. With that pop-up, the user will think “alright, why don’t I send a few messages back?” and this may end up with a smooth introduction. Therefore, we came up with an idea of the standard goal. That is to encourage new users to download the app on the first day and let them experience chatting with the coach and have learning experiences.
This leads to enhancing the guidance video on our top page. The prior guidance was more like all in one; there were many topics from the introduction of the course, how to use the app, and all other elements mixed and packed in just one video. Therefore, we divided it into categories, named them and displayed those names separately such as “introduction of the course”, “how to use the app” and so on. In addition to that, we altered the app to allow us to change the order of the video and organized a smooth traffic line to encourage new users to start using the app right after watching the introduction of the course.
Ikeda: This is how we organized the traffic line of the registration. Our continuous discussion for enhancement led us to devise how we should assign the coach because we thought there was still room to improve.
At the Coaching Course for University Entrance Exam Preparation, we ask new users to fill in the desired university upon registration. Based on that information, if a student wanted to study literature at Waseda University, for instance, we used to assign a coach who studied an art subject at a private university. In other words, we focused on assigning a coach with a matched category of the user’s desired university. However, as for G10-11 students, they are not so eager yet, and they haven’t often decided their desired university. Therefore, we found there were cases our previous matching was not so effective.
Also, we had had several users insisting on “having a male/female coach”. Therefore, we asked Nakanishi-san to extract some data based on a hypothesis; “does the first charging rate get any better, and does the number of users who continue the course increase if we assign a coach of the same gender as the user?” The result turned out to be 5% better if we assign a female coach to a female user. We changed our operation for G10-11 students and considered the gender of a coach as well as the user’s academic preference. This change increased the first charging rate by 5% only in 2 weeks.
Q: 5% is a big difference.
Ikeda: Yes, it was a surprising result for us. I suppose we can say that G10-11 students value affinity with the coach more than the targeting at the desired university. I think this was one of the reasons why it went successfully this time. 
Sharing the data with “funnel” and applying the measurement with the speed
Q: I’m sure taking measures based on a hypothesis is an effective way, but sometimes, hypotheses do go wrong. How do you adjust the direction on such an occasion?
Ikeda: We monitor the data every week and had a meeting with the entire team. If we saw the figures were not going up as we hoped, we would change our operation within a week or two. Of course, if the matter was development related, it took us a longer time, but this was how speedy we were.
Mitsuoka: I think one of our strong points was the fact that the length of our cycle when taking a new action was quite short because our team received feedback based on data and figures promptly. There are many kinds of measurements aiming at improvements, some of them would require changing UI of the product and/or a serious amount of development. Affluent resource of development is inevitable for these kinds of measurements. In our case, engineers who helped us were also assigned to different projects, so we considered how we could change the operation even though our developing resource is limited. This is how we came up with an idea of dividing and changing orders of the guideline video. I think our speedy cycle was realized partly because we looked for a light alteration without modifying the system, and received feedback regarding our hypothesis.
Q: Your attitude to discuss and decide the direction based on the data, and experiment it is really through, isn’t it?：
Mitsuoka: I think that is because we have this deep-rooted culture of “looking at the data” in our team. Also, the fact that everyone had an environment to look at the figures worked well thanks to Nakanishi-san because he had coordinated tools.
Nakanishi: We use a monitoring tool to feedback the activities of paid users and users on a free trial. By using this, I collect data such as the rate of users on free trial who pass through the point we would like them to pass during the first 2 weeks of the trial period. I use a funnel to make it visualize. We used this funnel to monitor the result of our enhancement of the guidance and assignment of the coach.
Ikeda: I think the culture of being fact-based is common in the entire StudySapuri. It became more so after Quipper joined the Recruit Group. Under the current circumstances where there are engineers in the office and everyone is cooperating to develop a product, our culture values looking at the data and discussing good points and things to be improved. Thanks to this, we managed to take measurements without too much effort. We knew the fact through the data Nakanishi-san prepared, so all we needed to do was to change the necessary operation.
Mitsuoka: I am really interested in how hard it was to make that funnel. Our project had never been successful without it. How did you coordinate it?
Nakanishi: I didn’t make it for this particular project. I had always thought that “it might be easier to enhance the business operation if we had had a tool like this, and kept brushing it up little by little. I had the base, so it was not such a hard job to make it if we knew what indicator to look at. Having said that, I needed to try and error to find out which indicators were effective in terms of the analyzing point of view.
Keys to success are a close relationship of the team and piling up small improvements
Q: What do you think was the key factor in the success of this attempt?
Nakanishi: Though I belong to the Data Team, I have moved to sit and work with the other members of this team. We sit next to each other, so we can easily communicate a tiny little problem and that seems to help us run the PDCA cycle quickly. To achieve a good result, I think it is very important to be able to communicate easily both physically and structurally.
Mitsuoka: As in the case of guidance video, I think there are things we can do even though the available resource is limited. When you say you are taking measures, you may feel you need to do it properly with abundant resources and finance. However, I think we were successful because we had chosen a method that allowed us to receive feedback many times and took actions based on the feedback many times. Finding out what we can do, devising it, and doing it in an as short cycle as possible- I think these were the shortcut to reach the solution based on a hypothesis. Once you have a solution, you can always spend the resource afterwards. I also learned there was a way to develop a strategy like this.
Ikeda: I totally agree with Nakanishi-san and Mitsuoka-san. I think these two points are the key factors of our success; one is to start it on a small scale and run it quickly, and the other is to have a close relationship within the team. A big measurement looks gorgeous and eye-catching, so many people may think it will work. However, I think to start a small measurement and keep improving it many times may be closer to the goal. On top of that, what one person can do as an individual is limited. I think it is important to build up a relationship with many people such as Nakanishi-san, Mitsuoka-san, other engineers and partners, so that communication can be closely done, and we can share the common recognition of valuing data.
Q: While you were working as a team, what have you learned from the other 2 members?
Ikeda: Nakanishi-san is a really fast worker. It is vital for us to look at the data, and he does it quickly and precisely. It has been very helpful. The communication skills of Mitsuoka-san is superb, so I always think he is suitable for a Product Manager. He has been a bridge between engineers and us, the business side team, so it is too bad to see him leaving the team….
Mitsuoka: It’s really nice of you to say that. To me, Nakanishi-san is not only a fast worker, but he also asks questions to improve accuracy such as “you want it with a definition like this, don’t you?” and “how would you like me to narrow down the duration?” whenever I ask some broad requests. I am truly grateful for that. As for Ikeda-san, although her role is on the business side, she thinks things really logically, and she talks including the background in order. This let me discuss the requirement of the development easily.
Nakanishi: To be honest, some people are not very good at dealing with figures on the business side. However, Ikeda-san doesn’t use data for guessing, but she actually utilizes them. She is determined to decide things after looking at the data, and she also gives us feedback from coaches. It was really great working with her. Mitsuoka-san used to be a developer, but he also has a sensitivity to the business side. He is approachable, and that has made the discussion involving the business side really easy.
Q: What would each of you like to do next?
Nakanishi: I would like to aim for maintaining the improvement of the first charging rate enduringly. To realize that, I would like to think things with the owner’s point of view, and have more ownership. I am assigned to the G4-9 Coaching Course as well as the Coaching Course for University Entrance Exam Preparation, so I am hoping to become a bridge of these two products and share my knowledge I gained in one product to the other, and vice versa so that both products can absorb the good parts of each other.
Ikeda: In the last fiscal year, we tried to match the gender of a coach and a user to improve the first charging rate. In this fiscal year, we are going to change the system of coaches into two groups; G12 and graduate students and G10-11 students. We have just started changing the logic we use when assigning the coach. I am half excited and half worried what sort of results we are getting, but I would like to analyze how much improvement we can see, and find new points to improve so that we can keep improving the product to provide a better learning environment to the user.
Mitsuoka: I was transferred to another team this April. I now belong to a team that provides StudySapuri to shool in the B2B business department. I think there is room to enhance by organizing data and facts in this team. I would like to use my knowledge I have acquired through my previous assignment and to start doing things like to promote maintaining a funnel, to establish KPIs within the team, and so on. I would like to share my knowledge of this project to the B2B team.
2020/4/15
* The business (including business name and business description), people and titles introduced in the article are those at the time of the interview. It may be different at this time.
