Hello, World – Zerodha, India's largest stock broker
zerodha.techThere is some reports that they sell your contact information to shady investment marketing companies. Many people have observed a sudden influx of calls from small, unheard of investment companies calling them up with investment offers, after they have signed up with Zerodha.
Also there have been much publcised issues with futures trading on their platform, with a court ordering them to refund a large sum to an invester.
There also have been issues with outages:
"Unfortunately, the steep increase in client base and consequent surge in orders has led to several bottlenecks because Zerodha’s infrastructure has not kept pace.
The system conks off often and it is common to find traders expressing their angst against Zerodha, especially on days of high volatility.
Yesterday was a textbook example of this.
The news about hostilities between India and Pakistan led to a sudden surge in volatility.
Traders scrambled to take new positions or cover their existing ones.
However, Kite, Zerodha’s flagship trading system, was down and out.
It stopped taking orders, which is an unthinkable eventuality given the mayhem that was being witnessed in the markets at that time."
From https://rakesh-jhunjhunwala.in/zerodha-held-liable-to-compen...
I've talked about the technical issues in a couple other responses on this post.
About spam, we've been working with the capital markets regulators, cyber crime units (Police), and TRAI (telecom regulator). It is a nasty issue that plagues all capital markets institutions in India. I'd gone into some detail on the data leaks on this Reddit AMA last year: https://www.reddit.com/r/IndiaInvestments/comments/bhv18e/we...
Ah..Ok, so contact information may have been leaked at various stages of the regulatory process or even at the Telco..
Good that you are taking action on the spam callers and hopefully other improvements have reduced the service interruptions. The regulatory environment possibily does not look with kind eyes on new entrants in the financial services industry. So I think you might have had many struggles to reach where you are.
I really didn't intend to shower negative sentiment about the service, only to highlight a few complaints I have heard, but later I saw it had become the topmost comment.
Wow, fascinating. You're being screwed by your service providers. Great job tracking that down.
In addition to a do-not-disturb list, can we have a nationally curated list of scamsters and aggressive ngos? The blacklist on my phone is now nearly 25 long.
As an analogy, it is precisely this kind of a distributed tracking and control of spam and DDOS that made cloudflare a valuable service.
That's just awful.These deep rooted issues plague so many good movements around our country.
Agree with the contact sharing thing. As soon as I registered I started getting telemarketing calls from many "Investment Research" companies giving me "recommendations".
> There is some reports that they sell your contact information to shady investment marketing companies. Many people have observed a sudden influx of calls from small, unheard of investment companies calling them up with investment offers, after they have signed up with Zerodha.
This might be only a single anecdotal data point, but I have been using Zerodha for more than a year and I have not received any spam call from any such investment companies. So whatever the spam call issue maybe, I doubt it is Zerodha selling your data.
I signed up couple of months ago and received no such calls.
All of our performance-critical, high throughput services are written in Go. We have not received any unsolicited advice asking us to rewrite everything in Rust or Nim (yet).
HN post is going to change that real soon..
Look forward to their 2021 tech post "How we migrated 344,245,587 lines of Go in 343,454,346 of rust and increased performance by 1.000034454%! /s
See Rust's correctness guarantees are going to help you avoid a full-on Robinhood situation ;) jkjk
I like the varsity website: https://zerodha.com/varsity/. It has very good educational content on Stocks, Currencies, Commodities Markets, and Investing. I really like the last chapter, Innerworth: https://zerodha.com/varsity/module/innerworth/
Thanks for sharing this.
Kailash Nadh, zerodha's co-founder, is an active investor as well through the rainmatter startup incubator [0]. There's been quite a boom in the fintech space in India for quite some time now, but zerodha is one of the first generation companies that survived through thick and thin. Indian fintech and edtech companies invite a tonne of interest from YC, in particular, and so, dare I say, the going has got a bit easier in the post-Jio, post-UPI world.
That said, I don't get the attractiveness of active-passive stock brokering. Indexed funds (mutual funds) remain a better investment vehicle for the casual passive investor [1] (though some doubt that [2]), as this famous bet [3] and this famous pre-IPO advice to googlers remind us [4].
Don't have a horse in the race, but in India, apart from coin.zerodha; groww.in and smallcase.com are two of the many new-age mutual / indexed fund companies.
[1] https://news.ycombinator.com/item?id=12768319
[2] https://news.ycombinator.com/item?id=20877700
[4] https://web.archive.org/web/20070417031443/http://www.sanfra...
Love what Zerodha has built. By opening up their platform they've allowed multiple startups to mushroom. As you pointed out, smallcase is one of them (I work there).
Thought I'd point out that Zerodha Coin, Groww allow for investing in direct mutual funds. On the other hand smallcases are quasi-ETFs. Basically a smallcase is a basket of stocks tracking an idea, theme, strategy, model.
I'm not an expert in trading, but I thought that you could only invest in the indexed/mutual funds available in your country?
I'm not sure your average Indian can, say, start putting money in any Vanguard fund available to Americans.
Sure you can. You just need a broker that lets you. Interactive Brokers for example.
There are thematic MFs that invest in say NASDAQ, S&P500 etc also some directly invest in US stock markets.
You can directly invest in NASDAQ and NYSE via https://greentiger.co/
It’s a YC funded company too.
Lol thanks. Love how their design is so heavily "inspired" by RobinHood :-)
This was an amazing read. Thanks for the post. While leading a really small tech team for a fast paced and ever-morphing world of Esports in India, this was a really helpful (It looks like your team plays a lot of CS:CZ. SoStronk is our product #ShamelessPlug). This point of yours caught my attention.
>Be extremely wary of technical debt. Know when to scrap and rewrite systems. We have scrapped and rewritten the majority of our stack, including our critical trading platforms, multiple times, improving them significantly with each iteration. These are tough decisions; extremely important trade-offs. Of course, non-interference from non-technical management is incredible luck.
This is what i have always struggled with. I fear the day someone asks me the version of a few web frontend libraries we use. Being debt free from day 1 is a bit farfetched for our industry. So the balance between "work on that tech debt" vs "build new features" always tips towards the latter. The philosophy till now has been to get it working and don't change what ain't broke. But make sure good engineering practices are always in place as much as possible. Still figuring out where and how do i invent time to fix tech debts and/or rewrite something
If you have a small team, don't spend too much time reading what a company with 1000 employees does. Decision making and risk calculus doesn't match. At all.
With small teams your job is to keep the factory lights on as long as possible, till the cash starts flowing predictably. If you have survived long enough for that, then you can think about tech debt.
Spot on.
If yours is a small team and if many things are keeping your team busy - never address a technical debt head on until you can afford the luxury or until the business demands warrant for it. Most engineering team focus too much on technical elegance and go overboard. Rewrite culture can become nasty if unchecked.
Yup. That's been the mantra for now.
Cheers. Not all libs need to be updated constantly either. Figuring out what is worth upgrading and rewriting is as important as rewriting itself. Also, it is okay to pause features sometimes to do an important rewrite.
Was able to a full remote account opening on the platform without a single phone call or courier of any documents
Customer onboarding + KYC is a big deal in India and this just surprised me as to how seamless it has become. My earlier "demat" account had simply so many hoops to jump through. Neatly done
I loved this ```Neither large teams for the sake of “growth”, nor 10x ninja developers, are meaningful. What matters is that a group of good developers, no matter how small, are able to work well together.``` As a software developer myself , couldn't agree more. Team members who can put of fires of each other , build really resilient organizations! , in your case it shows
Thanks :) The online on-boarding is the result of a multi-year effort of convincing the regulators of the advances in tech.
I am just amazed a 30 member team pulled of such an engineering feat. Great job , I am happy to see such strong engineering teams coming from India.
Agreed. I am mighty impressed.
> Minimal “AI/ML” for image and document recognition as an aid to operations.
Thank you, for not overhyping an ML deployment, and instead accurately describing the scope and purpose. Also seems like a very good fit for the problem space.
Too many times you'll see that section morphed into some weird marketing technobabble.
I literally highlighted every paragraph of this post. So many things to agree with.
I'm really looking forward to the breakdown of much of what has been teased here.
Top of mind: Is Postal the self-hosted transactional mail server (https://github.com/postalhq/postal)? Been looking at using this for an idea, and wanted to know if there are any obvious gotchas to look out for.
Postal is an amazing project. It just works out of the box. No much config required apart from the DNS. You can set the postal worker/thread count on the Postal config and Mysql connection count based on your machine and that would do. Finding the right numbers based on the CPU/RAM would be trial and error. We were able to achieve over 100 mails/sec on a 36 core system.
Their github issue tracker is pretty active: https://github.com/postalhq/postal/issues/
This was a discussion we had regarding scaling of postal: https://github.com/postalhq/postal/issues/697
On the same line, you should checkout https://listmonk.app/ by @knadh and we use it internally for sending email to our over two million users.
I hope all the positive replies here are from those who actually used Zerodha themselves over a period of time. When I tried it last year, the website was buggy and even saw HTML errors in response. I moved away to a traditional broker as a result of that.
As I mentioned in another response, sorry about that. We've had three incidents (intermittent degradation of service over the last two months that lasted 7 and 30 mins respectively). There were two similar incidents last year.
Not sure what you mean by "HTML errors", but apart from these unfortunate incidents stemming from legacy infra dependencies, there are no issues, and we cleared a billion trades last year.
Not sure if you noticed, the volatility over the last two months caused all top five traditional mega-brokers to go down for hours multiple times over several days. Of course, none of this should matter to end users, but I just wanted to say that a lot of these issues in the industry are because of the ageing underlying infra. Thanks.
I have been closely following the progress of Zerodha in such a complex market and also an early customer (not now) for them from the time of its launch. With some of the best competitions around like Sharekhan, 5paisa, and Karvy it was nice to see how they started with zero brokerage concepts like Robin. I did not know it is now the largest stockbroker in India. And really nice to see the tech blog starting. I thought it would be a larger team and using a different tech stack and never expected this. And yes, like-minded developers really add more value to the product
Recently came across Zerodha and had to set it up for a friend. Really good UX, and they've done a good job maintaining with the rapid tech progression.
Also, thanks for the ERPNext.org suggestion - sometimes I wonder how many open source gems I have yet to see. (No, please don't link me to one more awesome-X list :P)
ERPNext is an amazing piece of FOSS tech that deserves a lot more visibility.
There is also a high-tech stock broker for Chinese/HK/US markets - FUTU[1]. No tech blog yet. They have an open[2] Python API though.
A lot of good tips in there. Thank you for sharing. One thing grabbed my attention...
> Hybrid infra. Physical racks where numerous exchange leased lines terminate + AWS. Sometimes, these leased lines go down when the civic body in Mumbai digs up roads.
Isn't it better to fully rely on Cloud and Infra providers in order to completely avoid such instances?
My guess is that the leased lines are a requirement to connect to their partners.
That is correct. Have to have physical racks at approved data centres to terminate exchange leased lines. System audit regulations means that exchange approved auditors walk into the data centres and look at the racks and verify compliance :)
Cloud costs sky rocket when you have applications that use a lot of data. This setup looks like its a very good cost competitive decision
It would be great to see a discussion about using Flutter vs everything else they tried. Same wrt to Vue.
The first version of our web trading platform was written in Angular. It quickly got out of hand in complexity (and we felt Angular had a learning curve that was unwarranted for a frontend lib). The bundle sizes were also big. Then the Angular version fiasco happened. Vue turned out to be far easier to understand and work with, was faster in our benchmarks, and smaller in size.
With Flutter, we initially had a native Android app and a React Native iOS app. Managing two codebases was obviously painful, and React Native had significant performance issues, and we also struggled with the lack of stable libs, and the breaking changes between releases. Flutter came along and we were able to share 90% of the codebase across Android and iOS apps, and were quite happy with 60FPS rendering of native UI.
How did you find writing your apps in Dart when using Flutter? That was one language I haven't been in a rush to learn but the performance metrics you talk about can be quite persuasive.
I've heard a number of people having issues with React Native. I'm curious if that is just an immaturity thing or something more fundamental.
Dart is alright. For being able to use Flutter, it's a fair trade-off.
React Native, it was both. Fundamentally, it uses a JS engine that acts as a bridge, and that can be a performance bottleneck depending on the usecase.
Thanks for the response!
Flutter sounds like a good compromise between Native code (Swift/Kotlin) and React Native then.
If you need high performance but no code sharing - use Native code (Swift/Kotlin)
If you code sharing but okays performance - use React Native
If you need both high performance but code sharing - use Flutter
If anyone from Zerodha tech team is here, I wonder what's the reasoning behind Zerodha's weird 2FA setup? A static password + a static pin is not really 2FA IMO. What's the problem with supporting standard hardware or app based 2FA that requires an OTP?
Hey, you can turn on biometric 2FA on your smartphone, or use TOTP (Kite web -> My profile -> Password and security).
The static PIN is a legacy hole that has to be plugged. Coincidentally, we wrote to the regulator about this exact same matter recently. We'll hopefully see a regulatory announcement mandating "real" 2FA for everyone.
"The static PIN is a legacy hole that has to be plugged. "
...and probably never will be.
Bootstrapped, 10 years old and their tech looks great (from a customer UX perspective). Cool that they’re launching a tech blog. I’ve learned a lot from business engineering blogs over the past decade
New graduate here. Can you share some blogs which could help me ?
Very interesting. I haven't used this but it sounds very impressive. Cool takeaways. Nothing useful to share except that I enjoyed the write-up.
Interestingly, I've learned a lot about Indian operations from your comments regarding this (like telecom operators stealing and selling customer data, etc.). Fascinating!
Fantastic read and very elaborative.
Can you stop using third party trackers like google? I have network wide advt and tracker blocking with PI hole and the events links in the app doesn't load because googletagmanager domain is blocked.
They screw their customers atleast once every month with technical glitches resulting in huge loss for the day traders,who are their major customers..and they are never sorry about it.
Hey, sorry about that. And we're always sorry about technical issues when they happen, and 9/10 times, they stem from systems outside of our control.
The underlying infrastructure in the Indian capital markets is very constrained and was not really built for the kind of scale we're operating at in 2020. The next biggest broker processes only a fraction of our daily volumes. So it really is a matter of scale, industry wide.
To give you an example, the leased lines NSE (National Stock Exchange) offers to brokers to send orders take up to four months to commission, and each line has a max capacity of 1000 messages / second. We've been processing close to 8 million trades a day. So you can imagine how painful things can be.
We've been working hard over the years to reduce legacy market infrastructure dependencies and have been making significant progress.
Wait, so what's the problem here?
You're processing an average of 1.33 million trades/hour. Let's double that and round up for peak traffic, that's 3 million trades/hour or 833.33 trades/second.
Unless I'm missing something, you only need 1 line. What's the problem here?
That makes it sound like the only thing involved in stock broking setup is pushing messages through a line :) Market volatility can cause several hours worth of orders to come through in a single burst. What goes on inside a stock broker is actually incredibly complex. I guess that warrants a long blog post.
You bet, it does. With a case study would be interesting to read. At least that is worth for all the loses on that 1/10 because of your technical issues.
Saying sorry doesn't fix the losses. Pro traders are used to taking losses and can manage it. But it is super bad when we lose more than 50% of a huge profit in case of a exceptional market move just because of Kite not able to handle the load and accept an order. Or worst when there is an exceptional loss that just doubles because of system failure. You guys did a great job on the platform, but you should provide reliability to professionals who don't mind paying you extra for that
Thanks, but paying extra for reliability implies that reliability is being compromised and can be provided for a fee, which isn't the case here. As unfortunate as these incidents are, they are rooted in the legacy infrastructure a stock broker is dependent upon and we've been working hard over the years replacing those, component by component.
You became the largest broker, next may be you could become an exchange :D
What exactly are the legacy market infra dependencies and what would you replace them with? 8 million trades a day with 1000 messages / second doesn't sound too challenging. But it is besides the point w.r.t the user experience complaints people are raising here.
You can't send more than x orders in same line unlike traditional network where you can saturate the bandwidth.
Well that is so true, sometimes the glitches are very consistent across a few days and they take time to acknowledge it or will not acknowledge it. Over a few years I have come to understand what is reliable and what is to be double checked and when it is not reliable. I have ended up with bad order executions because of their buggy Kite Web UI that doesn't update at times. The UX for trading is not professional. Don't have to invent a new interface, just look at the existing desktop platforms. And they refuse to provide a desktop trading platform. The only good part is their apis so that people like me can develop their own interfaces. But once I find another broker with reliable REST APIs for OMS and lesser glitches, I will move out.
Good luck with that.
Are there any good resources for illustrating the technical components of a brokerage? I haven't seen many.
Zerodha.com seems to be banned in Tunisia. Is that because of spam or due to some regulations in India?
Pretty modern tech stack for a ten-year old company. Did they do many rewrites?
We did indeed. The most critical components have been rewritten and refactored multiple times as we have grown.
Do you guys provide bug bounty?
Yes they do provide bug bounty and I am still waiting for API credits from past 3 years for finding a bug in API, where anyone can use it for free.
It is great that they fixed it immediately after I sent them the details. Thanks to Nikhil Aralimatti (Business Analyst), who promised that the credits will be there in my account EOD. I should have asked if they live near a blackhole, because it looks like their EOD has not yet come for 3 years.
Apologies. This definitely would've been a genuine case of human error or oversight. There is no logical reason to not issue API credits that were promised. If you are still interested, please write to me and I will get the credits added.
Surely interested, if it's lifetime credits. Because, that's what I saved zerodha from everyone else using your service for free. Unless it is lifetime credits, I don't want to take pain migrating my integration to zerodha for few months of free credits.
I am sending you an email with more details.
We do. We have not yet documented this publicly, but will setup something like a Hackerone profile soon.