IPython gets $1.15M funding
mail.scipy.orgIPython notebook is actually one of the most interesting developments not just in the python world, but in computing generally. It's the first step towards the fully graphical shell on the internet that many of us have been looking for, whether or not we realize it.
Being able to weave together text|markdown, tabular data (in almost any format you want) and images ( whether from matplotlib, raw captures, or synthetic assemblies ) in one environment is very powerful.
I suspect that with some additional tools IPython notebook will become the integrators workbench par excellence, useful in many contexts.
It sorely needs some better tutorials, demos. The official docs are dry and dwell on technical matters rather than giving you some sense of the potential. There isn't much else out there - especially if you aren't doing scientific or statistical work.
I'm sure the iPython Notebook could be useful in my day to day Python/Django usage but I haven't figured out how yet.
There's a great (but somewhat lengthy) video tutorial[1] from Pycon US'12 that covers a lot of IPython and the Notebook. Highly recommended; it's a live demo by the authors of the tools. Seriously, IPython is the coolest thing that happened to Python since Python, and anyone even slightly interested owes it to himself to play with IPython.
Wow. Python has become something like Smalltalk, but with some architecture improvements. (Client/server through zeromq) This is to say, it's gone from programming environment to a multimedia glue that allows the integration of automation to everything. (Google DynaBook)
Thanks. Personally I'm allergic to video tutorials but I'm sure that will be great for others.
I'm interested in installing EPD, but it seems to want to install in my C:\Python27 directory (I'm on Windows here at work). Will this overwrite or otherwise interrupt my existing Python installation?
Hey, you should try the Anaconda Community Edition. It's a larger download, but that's because it comes with more packages(which may be good or bad, depending on your opinion) But it by default installs into C:/Anaconda, and it allows you to change the installation directory.
I work for Continuum, btw
As far as I can remember - yes, it will try to overwrite your Python dir, at least it did for me on my Windows machine (I backed up my original C:\Python27 before trying EPD, and then had to restore it, IIRC; there was also some mess with paths, but nothing incurable). I also tried installing EPD into its own directory, but Windows EPD package failed to run Notebook anyways (some issues with Tornado), and I was just too excited to try Notebook. Jumped to Ubuntu@EC2 and was running Notebook with EPD from it in 5 minutes.
Your mileage may vary, of course, so I'd say have a backup and give it a try - at least EPD didn't ruin anything for me. I'm pretty sure IPython Notebook on Windows machine is doable with some lib-hunting/building, since I managed to have fully functioning IPython Qt console installed and running perfectly on Win7 right now, and Notebook is just a couple additional requirements away from it.
The only libraries you should need to run the notebook are pyzmq and tornado, both of which you can download for Windows from Christoph Gohlke's page: http://www.lfd.uci.edu/~gohlke/pythonlibs/
That said, it's very easy to get it going on EC2, especially with Notebookcloud.
Sorry you got those issues with the notebook, this should not have happened. The notebook is certainly supported on windows in principle, the fact that it does not within EPD for you is a bug, I apologize for that.
With the latest EPD 7.3-2, ipython notebook --pylab inline works for me on windows 7 32 bits + chrome. Feel free to contact me privately (see my HN account) if you want more help on how to get this fixed.
Ryan, you should be able to select where it will install itself. It will override your existing python installation if you install it in the same location as your original, so don't do that.
I second this recommendation. Its an awesome demo. I used ipython from before, but barely scratched the surface. Now I use IPython to finish up assignments where the code documents itself, and with extra tex/images to fill in where required. I even 'cheat' sometimes on Project Euler by running problems in parallel using ipython. ;)
that is amazing. I assumed ipython was just another interactive shell - I will be using notebooks to collate my thoughts forever now! (indeed, it sounds like I could now do most of my development off a chromebook and a cloud server.)
edit: for clarification, I saw this comment, and three hours later I'm back to reply.
If there's one complaint to make about IPython, it's that the authors seem to be positioning IPython as a scientific/mathematical tool, even though it is so much more. Most of its docs assume you will be using IPython for its pylab and matplotlib features, instead of as a general-purpose Python shell. This is not surprising considering its roots, but still.
That said, I thought it was quite straightforward to setup and use the IPython Notebook.
We do make sure that it works without the rest of the Scipy stack, so it's useful for people outside the scientific world. But most of the core developers have a scientific background, and that's where we see the most interesting challenges.
In addition to the other links in this thread, Fernando's PyCon.ca keynote's has a great introduction to the power of IPython and how it's revolutionizing the scientific (and soon the general developer) communities. https://www.youtube.com/watch?v=F4rFuIb1Ie4
This is the best video I found, it has a demo at 20:00 which was recorded November 2012.
For anyone else who doesn't know what it is, this seems useful:
I hate to dump on this but IMHO what the world has been waiting for is not a Mathematica clone in Python. Neither do I think it would be a step in the right direction to funnel Python development through the browser.
There are those who will find it useful but others will find that there are better desktop tools for these purposes. Individual developers are free to pick and choose.
Fortunately, neither this announcement not the Ipython roadmap emphasises development of the web-based notebook.
Executable documents are powerful tools for teaching and learning. Focus in this area will pay off all across the Python community.
Executable documents are powerful tools for teaching and learning.
These already exist, though, and the ecosystem already has a lot of support and momentum behind it: javascript within the browser.
Would you explain what some of the benefits are in comparison to this? I'm an outsider so I don't have a very firm understanding of what the implications are.
I'm very optimistic about iPython as the solution for "executable documents", much so over Javascript by itself (despite all of Bret Victor's amazing efforts).
There is a development branch of iPython by Brian Granger that emits JSON when you display a python object in the notebook. You can easily create handlers that display any Python object using javascript. That means D3.js or whatever HTML5/WebGL app you can throw in there becomes the front-end to a very powerful numerical computation back-end. Check out this demonstration I help code, http://www.flickr.com/photos/47156828@N06/8183294725
Javascript isn't quite what people mean by 'executable documents' - this is about an interface which mixes visible code blocks and results with rich text.
You could certainly base something like that around Javascript, and it would have certain advantages. But the Python world has a lot invested in scientific computing - Numpy, Scipy, and dozens of packages built on top of them. It's also got good integration with other languages, from FORTRAN to R. This sort of scientific computing is where we think the notebook really shines.
As other replies have already eluded, this isn't really comparable to JavaScript. Also good luck crunching huge datasets on clusters from within your browser with a pure JavaSCript implementation.
Python has a tons of scientific libraries and scientific who know how to use them.
And for doing harm. (re macro viruses for MS Office).
What do you think the world is waiting for, and what are these better desktop tools?
Most of my programming these days is actually done in IPython notebook. Having an instant feedback is invaluable. I put my code in plain old .py files only once it needs to be imported.
I'm even wondering: if there was a way for notebooks to import each other, would I still bother with modules?
You can import notebooks in other notebooks if you turn on the option to autosave your ipynb files as .py files too.
1. In ipython_notebook_config.py, set the following:
# Save .py for each notebook for import elsewhere
c.NotebookManager.save_script = True
2. Restart your notebook server. 3. Save any notebooks you'd like to import under module friendly names. (They wind up in the same dir as the ipynb files.) 4. In your interactive notebook, import your_other_notebook.
This works well when all your notebooks are in the same folder, which is the default. I haven't given much thought to how you might create packages this way, but perhaps with a few code changes you could?
We don't really envisage the notebook as a tool for creating modules and packages. It's not a replacement for a traditional editor or IDE; indeed you could use it alongside one.
We might even pair it with a web-based editor, like ACE, so you can comfortably write modules on the same server.
To be fair, this graphical shell functionality has been part of Mathematica for at least the past 7-8 years. In fact, I would argue that within Mathematica, one can work with a much larger number of formats and data sources.
I think its important that this is coming to the FOSS world though. I agree that Mathematica/MATLAB are brilliant in this regard, but you (or your employer) has to shell out the big bucks for them, so if you want to collaborate with others who aren't on your platform you're almost certainly out of luck.
I was lucky enough to find IPython early on when I started with Python. I'm constantly amazed with its feature set. Some cool things I found useful:
The "run" command for running python scripts:
* -t to print timing information
* -p to print profiling info from profiler module
* -d run the script under pdb interactively (with -b to set line breakpoints)
* -n to set __name__
All of these things can be done fairly easily on their own, but doing it through IPython makes the output so much easier to read and work with.
Whenever I'm working with data I need to visualize, I like to use:
ipython qtconsole --pylab=inline
When I try to set __name__ with -n it says it's a bad flag. This is with the shell command. Is there something special I need to do?
You use it inside IPython, with a command like:
%run -n foo.py
You can't set __name__ to arbitrary values. It's either '__main__' like when running a script, or (with -n) the module name, like when performing an import.
IPython is extremely useful.
I like the qtconsole feature but i find it less than perfect to install:
1. pip install ipython 2. <system package manager> install qt4 3. pip install pyside / qt4 bindings
The better feature, for me anyway, is the notebook interface (it starts a local web server and gives you a rich web guy, somewhat like a Mathematica idea of interactive notebook editing with inline charting etc.)
But, that's a whole other bunch of dependencies that aren't in scope when installing ipython.
Not sure of the best fix, offer ipython and ipython-full-stack on PyPI?
If you want qt, I would say your best bet is one of the full python distribution for scientific python, e.g. winPython, python(x, y), EPD, Anaconda. Everybody working on one of this is a user of ipython, and care about the notebook, and most of them have a free version available (free as no cost, not necessarily open source)
(disclaimer, I work for Enthought, which sells EPD)
The free version of Enthought solved a bunch of problems for me on OSX. It bundles a lot of the binary libraries that are a bitch to compile yourself.
I've had a Mac at home since 2007 and had a go at doing some python programming on it a few years ago and gave up. I got 2.7 installed, virtualenv, even Qt but eventually a few things broke and I ended up switching to programming in a Linux VM.
I might have another go now I've got a shiny new Mini with ML on it. If i have problems I'll give Enthought a try.
Given the popularity of OSX with developers it's shocking how few Python libraries have good OSX binary packages. A few times I've found the easiest platform to work on is actually Windows. For example with PySide for a long time the binaries in all the Linux repos were out of date, OSX support was intermittently flaky, but the Windows installer was complete and up to date.
Yes I know you can compile from source, but I am a Python programmer. I develop software in Python, not C. I wonder if the poor state of binary packages for OSX is actually because it's a Unix variant. i.e. module developers deliberately choose not to do binary OSX builds because hey, the users can just compile from source. No, we can't all compile from source. We're not all C hackers, that's why many of us use Python in the first place. </entitled rant>
(disclaimer: I work for Continum Analytics)
Anaconda Community Edition installs very easily on windows, linux, and OS X, and is very easy to uninstall without breaking your system.
> No, we can't all compile from source. We're not all C hackers, that's why many of us use Python in the first place.
EXACTLY!
This is the whole idea behind Anaconda and the idea behind our beta cloud product Wakari. Wakari gives you multiple python versions, multiple numpy versions, interactive plotting in your browser, and the wonderful ipython notebook built-in. It's still in a closed beta, but people usually get approved for access the same day.
Glad it is working for you. I have now just started working on it, so let me know if there are things you'd like to see improved.
That's why for apps like this (not for projects that I create with virtualenv) I much more prefer using OS package manager. Could it be any nicer than on Gentoo? ;]
CDEPEND="dev-python/decorator dev-python/pexpect dev-python/pyparsing dev-python/simplegeneric virtual/python-argparse emacs? ( app-emacs/python-mode virtual/emacs ) matplotlib? ( dev-python/matplotlib ) mongodb? ( dev-python/pymongo ) octave? ( dev-python/oct2py ) smp? ( dev-python/pyzmq ) wxwidgets? ( dev-python/wxpython )" RDEPEND="${CDEPEND} notebook? ( >=www-servers/tornado-2.1 dev-python/pygments dev-python/pyzmq ) qt4? ( || ( dev-python/PyQt4 dev-python/pyside ) dev-python/pygments dev-python/pyzmq )" DEPEND="${CDEPEND} test? ( dev-python/nose )"For OSX, I've used http://fonnesbeck.github.com/ScipySuperpack/ with no issues.
I'm sure it's useful, but is it extremely useful? I can imagine living without it, even when programming Python.
I'm surprised to be honest but maybe I'm missing something.
>> I'm surprised to be honest but maybe I'm missing something.
Give it a proper whirl, you'll never go back.
Proper python REPL history across sessions (not hacking up a substitute using a $PYTHONSTARTUP file to load readline history and register an at exit handler to dump history). History becomes as useful as a large shell history (i'm sure i did something similar last year, what was it?...)
In line charting, makes visualising any random thing (runtime of function, memory usage etc.) so easy you'll start doing it all the time.
The webserver notebook can stream line collaboration with co-workers. I like the ability to make "interactive documentation", although truth be told i haven't used this as much as i could yet.
My first shell was csh. I put off switching to tcsh, but eventually did. After a while I grew used to the new features of tcsh, with tab completion being the most obvious. I then went back to csh on some machine and realized just how dependent on was on tab completion. It became frustrating to type all of the filename again, rather than just the first few characters<tab> follows perhaps by some another character+<tab> for disambiguation.
I don't want to go back to a shell without tab completion. Is tab completion extremely useful? Functionally, no. I can live without it. But I will avoid that if at all possible, and that's not something I felt before I used tab-completion.
I get the sense that IPython is the same way. It's not essential, but after many have used it they have the sense that they don't ever want to be without again.
REPL history? Tab completion? Syntax highlighting? Unix shell integration? And that's just the tip of the iceberg useful for general Python programming, nevermind the exploratory scientific environment with numpy/scipy/matplotlib/pandas.
I can imagine living without it in the same way I can imagine writing code in Notepad. Technically feasible but unacceptably painful.
That grant is well deserved. IPython is easily the most useful Python tool I have ever used.
Absolutely - it is definitely essential to python development.
Although I hear more and more people talk about bpython[1]. I understand why it's different, but it's just not as awesome as IPython, especially since IPython 0.11+
This is big news for the Python community :)
The notebook interface is the natural generalisation of the REPL, and has the potential of being reused and copied by other languages (clojure, JavaScript) just as, in offline form, it has been used in Mathematica and Racket for years.
I think the notebook will eventually support other languages behind it.
It should do, yes. The developers of Julia have said that they're going to use the IPython notebook rather than rolling their own interactive interface.
You can try IPython (shell) and IPython Notebook (web interface) online with Wakari at https://www.wakari.io - platform (with all popular Python scientific libraries installed) that lets you run multiple Python sessions and access them online anywhere. It also allows IPython Notebook sharing - you can share your workflow.
There's NBCloud (http://notebookcloud.appspot.com) if you want to host notebooks on your own Amazon cluster (great for using the parallel computing features).
Also, PiCloud (http://www.picloud.com) has a notebooks tab for launching an IPython Notebook as a job. That's useful because it means you can use their compute and data facilities with it easily.
Why does IPython get all the love? I tried it out back in the day, but it was not very impressive. bpython is much better, but doesn't seem to get as much attention.
Do you work in scientific computing? I think if you did, then you would understand
- Will bpython allow me to control a cluster of python processes to perform my scientific computation?
- Will bpython allow me to create executable documents mixing markdown as well as LaTeX so that I can display mathematical equations?
- Does bpython have integrated matplotlib support, so that the GUI thread of matplotlib runs in a separate thread and won't block the interpreter?
IPython is much more than just an interactive shell, even though the name doesn't suggest more than that.
I know little about bpython. I have observed that IPython has very good outreach. Fernando Pérez has promoted IPython at many conferences, helped a lot of people, been responsive to requests, active in the Python community, and all of the other factors that help make a successful project.
It's also been around longer than bpython. IPython started in 2001 while bpython is from 2009.
The current IPython has several GUI interfaces, including through a web browser, with the goal of supporting a graphical notebook with embedded plots and other images. bpython does not support that.
There are very few tools in the extended REPL space. I think software developers don't spend as much time on the interactive shell as computational scientists, and those developers that do may augment the shell with, say, the emacs shell rather than develop a new interactive interface. So an early tool which works reasonably well, which has been around for a while, with few competitors, and has outreach and updates based on user needs, will tend to be more widely used.
How long ago was 'back in the day'? Development has really picked up since about 2011, and it's become much more than just a shell. IPython has a particular focus on scientific computing, and how we can mix ad-hoc exploratory computing with recorded, repeatable computing, which leads to tools like the notebook.
If bpython does what you want, that's great. But the equivalent functionality is just one part of IPython.
TIL there is a bpython.
Thus (anecdotally) proving your point, I suppose.
bpython is a very nice lightweight replacement for the standard Python interpreter. Nothing but <3 for bpython. But it's not really comparable in scope and capabilities to IPython; the latter is much more heavy-weight but for myriad good reasons.
IPython has history search, like bash has bash_history. A simple invaluable feature that bpython do not have.
This is the awesomest news for the day. It is so much more than a tool purely for the scientific community. I use it for quick fire data analytics codes all the time. Especially when coupled with pandas and matplotlib. pandas can output data in a tabular format directly onto the HTML notebook.
If creating forms on the same was a bit more simpler, then it would be biggest boon to a data analyst since excel sheets...dare i say. It would lead to quick fix applications, rapid prototyping for reporting applications. And it can be hosted and shared directly: http://nbviewer.ipython.org/
What's IPython?
It's a python repl, it has a bunch of useful syntax extensions such as autocall(not needing to type parens after no args calls and around one arg calls) ? ?? for help summary, syntax higlighting, readline, tab completion, optional integration with GTK, Qt or wxWidgets toolkits(ie letting you build and interact with guis in repl).
ipython has a client/server model with multiple gui frontends including a qt one that lets you print out inline graphs and ipython notebook which lets you generate documents with code and data analysis/visualization. I believe the last part is what the grant is for.
Python has been used in scientific and statistical fields a lot lately and ipython notebook is quite useful for that.
http://ipython.org/ipython-doc/dev/overview.html
Basically it's the Python terminal on steroids.
It is now much, much more than that. Besides the notebook (that allows you to mix python, images, math, etc... inside an html browser: http://ipython.org/ipython-doc/stable/_static/notebook_specg...), you get a powerful engine capable of distributing computation between different machines.
I would be wonderful if we could re-use the notebook interface for other language - Octave, Julia, R, Ruby.
(Would also be nice if building ipython on OS X was easier!)
AS I reply to another comment it is possible and on it's way.
for ruby cf https://github.com/ipython/ipython/pull/2643
(Would also be nice if building ipython on OS X was easier!)
What issue do you face? Genuinely curious, because I use Mountain Lion, and installing IPython for me simply involves doing
I installed Python from source though.pip install ipythonThanks. It mostly worked. Of course, the real benefit of ipython is the notebook. For that you need (on OS X)
Actually, for qtconsole you need to install Qt and pip install pysidebrew install python # or just download and install from source pip install numpy pip install matplotlib pip install ipython[zmq,qtconsole,notebook,test]But the latter failed to build on my machine... (Given that Apple's gcc version is ancient and clang is the default compiler on OS X -- it's a wonder the other tools were built correctly)
One of the many features of Org-mode in Emacs is that it functions in a similar-style to IPython's notebook for over twenty different languages, Octave, R, Ruby, Python, C, C++ and others: http://orgmode.org/worg/org-contrib/babel/languages.html
The related features in Org-mode are known as Org-babel, uniting disparate languages under the single umbrella of Org-mode. The support for language evaluation in Org-mode is slightly more cumbersome than that offered in IPython, but also arguably more flexible and powerful. Plus you get to edit your notebook in a powerful editor with best outliner ever, plus all of Org-mode's many other features (I'm a Vim person but I give credit where credit is due).
Org-babel is a rich and deep niche of Org-mode, and one of the areas under the most constant development. You can read some docs on it here, but it can be hard to wrap your head around all of it until you actually use it:
http://orgmode.org/manual/Working-With-Source-Code.html#Work...
Sage Notebook interface can be used for Python, R, Octave, Matlab, Mathematica, LaTeX among others. http://www.sagenb.org
Rmagic and octave magic work pretty well in my experience (which rely on Rpy2 and oct2py)
If I'm recalling Fernando's talk from PyConCA correctly, recent versions of IPython are already designed to support other languages: they talk to a python kernel over a wire protocol, instead of directly embedding the interpreter.
A critical component of the grant application was the proposal to make the notebooks extensible, with concrete plans to work on integrating R, and a strong desire to integrate with Julia.
SciPy Superpack easily installs iPython, SciPy, NumPy, Matplotlib, and others on OSX.
When will we hear about PyPy getting a similar amount of funding?
It's a great project, and lack of funding has prevented them from delivering on Py3k.
PyPy had 1,353,000 euro in EU funding, so they have received a similar amount already. However, pypy is also a much more ambitious project, and needs a lot more funding than IPython to achieve its full goals.
hand-written thank you from me is going out tomorrow to the Alfred P. Sloan Foundation (http://www.sloan.org/contact-us/)
It was great to click through and learn that the funding was a grant. With that kind of a number, I thought they had raised a VC round - not that companies working on open source projects is bad, but it is always nice that they don't have to worry about a business model.
This might be a silly question, and perhaps not the place for it, apologies in advance.
Love iPython, except for one thing I can't seem to figure out.
print "Hello World"What is going on?! I don't get how this split is supposed to work and it always throws me in it when I least expect it. I just want to do a simple edit to my previous command :(
Are you on OS X? IPython is probably finding libedit instead of readline.
Try doing an `easy_install readline` before you `pip install ipython` in your virtualenv. It should work just fine, then.
Do you have readline installed? Your sequence works as expected for me.
Thrilled to see this. IPython Notebook has become my go-to tool for data munging and analytics. I look forward to seeing the IPython team take it to the next level.
Fantastic news, IPython deserves it and can't wait what they will deliver next.
I'm surprised Python doesn't get more in return. Google, Pinterest, Disqus, ... all these guys have buku funding and use Python heavily. Why not show Python some love? Make the VM up to par with V8.
Congratulations, guys. Excellent work and I love Ipython :)
Awesome news! ipython + notebook is amazing.
I really wish there was an equivalent of IPython in Ruby; it's one of the tools I miss the most from python.
Hi,
The notebook does not care wether or not the language is python,ruby.. or else. As we are mainly python developer the notebook is known for it's python bindings, but the ability to run ruby code is on its way.
Cf instruction here https://github.com/ipython/ipython/pull/2643
We really would love ruby devs help into packaging and make it seamless to run for user, and any other languages that would wish to join.
I actually working on writing a Julia kernel which will be my first encounter with Julia.
There is pry for ruby
Congratulations, well deserved
Fantastic!