Show HN: Pixie, Instant Kubernetes-Native Application Observability Using EBPF
pixielabs.aiHaving used an earlier version of this, I highly recommend it if you're carrying a pager in micro-service land, and don't have Brendan Gregg on your team.
I love the idea of instrumenting my clusters with less work. Signed up for the BETA and starting to get data out of one of my GKE clusters right away. Also I like how the cli just interacted with the cluster itself instead of tons of piping the output to other commands. Good luck Pixie team!
Would love to see intrumentation without any code changes. Is it possible to write EBPF based tracers for JVM and Node/Python interpreters?
Co-founder/CEO of Pixie here.
EBPF allows you to access static tracepoints that are defined in many runtime, and can be used to capture information about the state of the runtime.
Since most VM’s/runtimes allow monkey patching you can usually get to the same level of information without using EBPF. We plan to add support for this in Pixie in the future and provide a seamless experience regardless of what underlying tracing technology is used.
Lots of good stuff from Brendan Gregg here: http://www.brendangregg.com/Slides/Velocity2017_BPF_superpow...
Great to see that zasgar. Monkey patching is pretty gnarly to replace functions at load time or in JVM/.NET CLR. Python is easier. Every version change, the monkey patch has to be updated. But you can get a lot of good value from it in the form of performance charts and call graphs of an application. It would be great value if you could reduce the gnarliness of the monkey patching work and possibly replace it with a simple configuration file. Have to look into eBPF in more detail.
Founding engineer at Pixie here.
To add to what zasgar mentioned, I just wanted to point out that our instrumentation-free approach does apply to JVM and Node/Python applications for many of the traces we gather. For example, we use EBPF to trace protocols like HTTP as the data passes through the kernel. By gathering the data in the kernel, these EBPF tracers are completely language agnostic.
What do you do for HTTPS which would be encrypted as it passes through the kernel and is only decrypted in the application process?
Great question. You're right that tracing in the kernel doesn't work for encrypted traffic (that means anything over TLS, including HTTPS). For encrypted connections, we still want to give the no-manual-instrumentation-required experience to our users, so what we do is trace the SSL/TLS library to capture the traffic. Right now, for example, we trace traffic going through OpenSSL. This has the benefit of covering a wide array of programs in different languages, including any dynamic languages that use OpenSSL. And we plan on adding more TLS libraries soon (e.g. GoTLS) to fill in the gaps.
We'll be publishing a blog post on this soon, so please stay tuned. In the meanwhile, this other post (https://docs.pixielabs.ai/tutorials/simple-go-tracing/) gives an idea of how one can use EBPF user-space probes to trace applications and system libraries.
Check bcc-tools: https://github.com/iovisor/bcc/tree/master/tools
this is really amazing. the marketing page is also top notch from a visual perspective
Whats with .ai for an ops tooling?
Co-founder/CEO of Pixie here.
At Pixie, we are building a new type of data system that can deal with the data volume that we collect and process. We have AI/ML models to help classify traffic and make analysis easier. Over the next few weeks, we will release some blog posts that will discuss how this works.