Enable anyone to process terabytes of data in minutes.
Burla makes it trivial to run Python functions on thousands of computers in the cloud. It's a Python package that only has one function:
This realtime example runs my_function on 1,000 separate computers in one second!
Burla is simple, flexible, and scalable:
Scalable: See our demo where we process 2.4TB of parquet files in 76s using 10,000 CPUs.
Flexible: Runs any Python function, inside any Docker container, on any hardware.
Easy to learn: Burla is just one function, with two required arguments.
In addition Burla is open-source, and can be deployed in your cloud using just one command. The included web platform makes it easy to monitor long-running workloads or data-pipelines.
Burla only has one function: remote_parallel_map
When called, it runs the given function, on every input in the given list, each on a separate computer.
Running code in the cloud with Burla feels the same as coding locally:
Anything you print appears in your local terminal.
Exceptions thrown in your code are thrown on your local machine.
Responses are quick, you run a million function calls in a couple seconds!
📦 Automatic Package Sync
Burla clusters automatically (and very quickly) install any missing python packages into all containers in the cluster.
Easily run code in any Docker container. Public or private, just paste an image URI in the settings, then hit start!
Need to get big data into/out of the cluster? Burla automatically mounts a cloud storage bucket to ./shared in every container.
⚙️ Variable Hardware Per-Function
The func_cpu and func_ram args make it possible to assign more hardware to some functions, and less to others, unlocking new ways to simplify pipelines and architecture.
Build massive pipelines with plain Python:
Create pipelines that fan in/out over thousands of machines, then aggregate data in one big machine. The network filesystem mounted at `./shared` makes it easy to pass big data between steps.
There are two ways to host Burla:
In your cloud. Burla is open-source, and can be deployed with one command (currently Google-Cloud only). Click here to get started with self-hosted Burla.
In our cloud. First $1,000 in compute spend is free, try it now 👇
Get a free proof of concept: We build first. You decide later.
Schedule a call and we'll build a functional proof of concept specific to your problem. We've delivered plug and play, production-ready systems for companies working on:
DNA analysis / genomic pipelining
Medical image processing for ML
Video game replay analysis
Reinforcement learning simulations
Industrial CT-scan analysis
Questions? Schedule a call, or email [email protected]. We're always happy to talk.
from burla import remote_parallel_map
my_inputs = [1, 2, 3]
def my_function(my_input):
print("I'm running on my own separate computer in the cloud!")
return my_input
return_values = remote_parallel_map(my_function, my_inputs)from burla import remote_parallel_map
# Run `process_file` on many small machines
results = remote_parallel_map(process_file, files)
# Combine results on one big machine
result = remote_parallel_map(combine_results, [results], func_ram=256)