Your path to Imply is an interesting one — please tell us about your academic background and how you began working for the company.
I studied computer science at Korea University in Seoul. My major was in distributed systems, an area I’ve always been interested in. It’s long been a dream of mine to develop my own distributed database.
About five years ago, I went to Singapore for an IT conference and attended a breakout session run by the founder and CEO of Imply, Fangjin Yang. Imply was a small startup back then and Fangjin was traveling around to get people interested in Druid. I had a lot of technical questions and he was very good at answering them. As I learned more, my interest in Druid only grew. I got really excited about all its cool features and the challenging problems it can solve.
When I returned to Korea, I contacted Fangjin and told him that I really wanted to join Imply. Even though I expected some challenges due to my limited English skills and being located in Korea, they brought me onboard anyway. I worked remotely from Korea for the first year before moving to California in 2018.
Tell me about your job and what problems you are trying to solve.
At its core, data analytics is about repeating query execution and result analysis. Data analysts run some queries, get some insights out of the results, and run more queries to get a deeper understanding.
The problem today is that data analysts are often unable to focus on their primary job — analyzing data — and instead need to spend a lot of time on tuning their queries. As they are not experts in the underlying system, it often becomes a very hard process for them that takes a long time. As a result, in the worst case, they could miss a golden business opportunity and even negatively impact their company.
This is the problem that we’re solving at Imply with Apache Druid — one of the fastest query-processing engines in the world.
“We are trying to make machines understand what humans need, so that we can make our customers happy.”
There are many companies trying to solve this problem, but a lot of the solutions out there still require too much manual work. What we are trying to do is automate the entire analytics process. I know this sounds impossible, but it’s our guiding vision. We are trying to make machines understand what humans need, so that we can make our customers happy.
Can you tell us about a specific project you’re working on?
Druid is an open-source project, so I also use my spare time to make Druid better. There is a side project that I’ve been working on about query speed.
One of the current challenges with Druid is that when a query has to read a lot of big data, Druid uses multiple threads to process that data faster. When you use multiple threads, you always need to think about how to synchronize and coordinate them for the greatest efficiency. So I have been trying to minimize the synchronization cost while maximizing the CPU utilization of those multiple threads. It’s very experimental work, but I’ve proven that my idea could generate speeds up to two times faster than what we have now. Now I’m working on making it more stable.
What’s it like to work with the team and how has it evolved?
About five years ago, when I joined, there were only four people in my team. Now we have 30.
We’re growing so fast that we’ve had to split the team into three smaller groups. All the new people are amazing and have really good backgrounds. The team is so welcoming and competent that people do not even need that much time to get adjusted — they just immediately start working on something cool.
“In the smartest team, you don’t need to be the smartest person — you just need to contribute with something that you’re an expert in.”
What we want to do is build the smartest engineering team in the world. This means thinking together, making decisions together, and growing together. In the smartest team, you don’t need to be the smartest person — you just need to contribute with something that you’re an expert in. This is the kind of team I want to be part of.
What can you say about relocating to the United States for Imply?
There is so much I could say — this company has really helped me a lot!
One of the things I appreciate the most is that everyone at Imply has always respected me. My English is still not perfect, and five years ago it was not as good as it is now. They could probably hardly understand me! But they always listened to me and gave me the time I needed to finish what I wanted to say or explain. Putting myself in their shoes, I’m not sure I would have had the same patience!
I’ve always been warmly welcomed to join social occasions outside of work too. It’s been a while now since I’ve seen everyone, as we’ve all been working at home during the pandemic. So I’m looking forward to meeting up again when the time is right.