Clojure at BackType

4 min read Original article ↗

More Related Content

PHP Backends for Real-Time User Interaction using Apache Storm.

The Secrets of Building Realtime Big Data Systems

Storm: distributed and fault-tolerant realtime computation

Learning Stream Processing with Apache Storm

Storm: Distributed and fault tolerant realtime computation

What's hot

Yahoo compares Storm and Spark

Real-Time Big Data at In-Memory Speed, Using Storm

Using Simplicity to Make Hard Big Data Problems Easy

Real-time streams and logs with Storm and Kafka

Real-Time Analytics with Kafka, Cassandra and Storm

Storm: The Real-Time Layer - GlueCon 2012

Realtime Analytics with Storm and Hadoop

Assigning Responsibility for Deteriorations in Video Quality with Henry Milne...

Scaling Apache Storm (Hadoop Summit 2015)

Slide #1:Introduction to Apache Storm

Analysis big data by use php with storm

Storm Real Time Computation

Introduction to Apache Storm - Concept & Example

The inherent complexity of stream processing

Multi-tenant Apache Storm as a service

Viewers also liked

Yet another startup built on Clojure(Script)

Clojure: Towards The Essence of Programming

Getting started with Clojure

Functional Reactive Programming with Kotlin on Android - Giorgio Natili - Cod...

Similar to Clojure at BackType

Realtime Analytics with MongoDB Counters (mongonyc 2012)

Is NoSQL The Future of Data Storage?

NOSQL, CouchDB, and the Cloud

Slide presentation pycassa_upload

Spring one2gx2010 spring-nonrelational_data

Sep 2012 HUG: Apache Drill for Interactive Analysis

Drill Bay Area HUG 2012-09-19

Drill at the Chug 9-19-12

Polygot persistence for Java Developers - August 2011 / @Oakjug

Intro to Big Data and NoSQL

Hadoop Summit - Hausenblas 20 March

Understanding the Value and Architecture of Apache Drill

PhillyDB Talk - Beyond Batch

Outside The Box With Apache Cassnadra

Codemotion 2017 - "Dime cómo manejas tus datos y te diré qué clase de base de...

More from nathanmarz

Demystifying Data Engineering

The Epistemology of Software Engineering

Runaway complexity in Big Data... and a plan to stop it

Become Efficient or Die: The Story of BackType

Cascalog at May Bay Area Hadoop User Group

Clojure at BackType

  • 1.

    Clojure at BackType Howwe learned to stop worrying and love the parentheses Nathan Marz BackType @nathanmarz

  • 2.
  • 3.

    APIs • Conversational graphfor url • Comment search • #Tweets / URL • Influence scores • Top sites • Trending links stream • etc.

  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.

    Cascalog Cascalog Variables and logic Abstraction Cascading Tuples, data workflows Key/value pairs, MapReduce aggregation

  • 19.
  • 20.
  • 21.
  • 22.

    Cascalog basics Where to emit results Define and execute a query

  • 23.

    Cascalog basics Where to emit results Output variables Define and execute a query

  • 24.

    Cascalog basics Where to “Predicates”: constrain emit results the output variables Output variables Define and execute a query

  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.

    Graph Schema Reshare: true Gender: female Property Tweet: 456 Property Reaction Reactor Reactor Tweet: 123 Alice Bob Property Property Content: RT @bob Content: Data is fun! Data is fun!

  • 45.

    ElephantDB Shard 0 Shard 1 Shard 2 Distributed Key/Value pairs Shard 3 Filesystem Pre-shard Shard 4 and index in Shard 5 MapReduce Generation of domain of data

  • 46.

    ElephantDB DFS ElephantDB Server Shard 0 Shard 1 Shard 2 ElephantDB Server Shard 3 Shard 4 Shard 5 ElephantDB Server Serving domain of data

  • 47.
  • 48.

    Stream processing • Automaticallydistributes computation • Horizontally scalable • Fault-tolerant • Guarantees processing of messages

  • 49.
  • 50.
  • 51.

    Tweets What is a query? # Tweets for a URL

  • 52.

    Tweets What is a query? Influence Score for a person

  • 53.

    Raw data Computing a query Fully precompute view DB Query

  • 54.

    Raw data Computing a query Do a live compute from scratch Query

  • 55.

    Computing a query DB Raw data Precompute subviews Compute query from DB Query intermediate dbs DB

  • 56.
  • 57.
  • 58.
  • 59.

Editor's Notes