What's up Python? Django get background tasks, a new REPL, bye bye gunicorn
bitecode.devWhat I love about Django is that you can create a Django project with just one file.
You can turn a fresh Debian machine into a running Django web app by just doing:
apt install -y python3-django apache2 libapache2-mod-wsgi-py3
/var/www/mysite/mysite/wsgi.py:
import os
import django
from django.core.wsgi import get_wsgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.wsgi')
application = get_wsgi_application()
ROOT_URLCONF = 'mysite.wsgi'
SECRET_KEY = 'hello'
def index(request):
return django.http.HttpResponse('This is the homepage')
def cats(request):
return django.http.HttpResponse('This is the cats page')
urlpatterns = [
django.urls.path('', index),
django.urls.path('cats', cats),
]
/etc/apache2/sites-enabled/000-default.conf:
ServerName 127.0.0.1
WSGIPythonPath /var/www/mysite
<VirtualHost *:80>
WSGIScriptAlias / /var/www/mysite/mysite/wsgi.py
<Directory /var/www/mysite/mysite>
<Files wsgi.py>
Require all granted
</Files>
</Directory>
</VirtualHost>
If you want to try it in a docker container, you can run
docker run -it --rm -p80:80 debian:12
Blown away this is possible. When I was starting to learn to code and was overwhelmed when learning all the bits and pieces of rails, I wondered if everything necessary to run a very small app could be placed in a single file. I guess this is the answer, but in python/django instead of ruby/rails. It would be a fun exercise in ruby/rails too (not totally sure if it's possible, but suspect it may be, albeit probably resulting in a larger file than the one you give above!)
Not that I think this is pratical or even looks good, but its also doable with rails
For very small apps on ruby land sinatra and roda are the right choices. On python the choice would be flask instead of the single file django.require 'bundler/inline' gemfile(true) do source 'https://rubygems.org' gem 'rails', '~> 7.1' gem "sqlite3", "~> 1.4" end require 'rails' require 'active_record/railtie' database = 'app_development.sqlite3' ENV['DATABASE_URL'] = "sqlite3:#{database}" ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: database) ActiveRecord::Schema.define do create_table :my_table, force: true do |t| t.integer :my_table_id end end class App < Rails::Application routes.append do root to: 'home#index' end end class HomeController < ActionController::Base def index render inline: 'HOME' end end App.initialize! run AppIs the db mandatory in Rails or could the db specific lines be removed from the code if no db is needed?
Not mandatory, you could remove that
Cool. What would be the steps from a fresh Debian to get it running and see it in action?
Not sure debian has (a modern) ruby installed by default, but I think you only need to have a working ruby installation and running the script. On a sidenote, this came out recently and could interest you https://dashbit.co/blog/announcing-phoenix-playground
I tried it like this:
I then I got:1: apt install ruby 2: Put your code in test.rb 3: ruby test.rb
So I tried:Gem::Ext::BuildError: ERROR: Failed to build gem native extension. \e[0m current directory: /var/lib/gems/3.1.0/gems/stringio-3.1.1/ext/stringio /usr/bin/ruby3.1 -I /usr/lib/ruby/vendor_ruby -r ./siteconf20240626-4044-iylnuy.rb extconf.rb mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h You might have to install separate package for the ruby development environment, ruby-dev or ruby-devel for example.
Then I got:4: apt install ruby-dev 5: ruby test.rb[4147, #<Thread:0x00007f9a4af6bc60 run>, #<NameError: uninitialized constant Gem::Source (defined?(@source) && @source) || Gem::Source::Installed.new ^^^^^^^^
This is also easily done by design with flask or quart, and probably several other frameworks.
There is like 10 python frameworks which achieve that with much fewer steps.
Yes but how do you go about defining an ORM model without an app init class?
I hanen't touched Django in some time, but something like this should work:
from django.db import models from django.db import connection class MyModel(models.Model): name = models.CharField(max_length=255) class Meta: app_label = 'myapp' with connection.schema_editor() as schema_editor: schema_editor.create_model(MyModel) # from django.core.management import call_command # call_command('makemigrations', 'myapp') # call_command('migrate', 'myapp')A lot could be said about it, but the first thing that comes to mind is that not every application uses a db.
And not every application that uses a db needs an ORM.
You probably can build an ORM into the single file. But I'm not saying every application should be developed as a single file forever. I just like that you can start with one and expand later.
Then you don't need django, flask and fast API already does all that withing a single file.
What makes django useful is the admin, auth, session and so on that are all built around the ORM.
You never know into what a project will develop. So it is nice to know you can expand it into anything, even when you start simple.
I don't catch if you are sarcastic or not
do you really consider this a lean process in 2024?
You get downvoted but my first thought was similar, I wasn't sure the author was being sarcastic or not, although for a slightly different reason: the whole process, if done more properly, has a few more steps (and of course even more for production) - but on the other hand it might be interesting for someone to know you can skip some and start a smallish app a bit quicker although I'm not a fan of such hacks.
This is one of the core criticisms of Django and python web backends.
WSGI is old tech that in your example requires extra packages and code in project to work. By now Python should stand on its own and handle this independently, being proxied to by better http servers like nginx or caddy.
Django supports ASGI: https://docs.djangoproject.com/en/5.0/howto/deployment/asgi/
You have to install software for software to run on your computer?!?
You have to install extra software that should’ve been part of std, big difference.
Are you still talking about Apache here?
> This is one of the core criticisms of Django and python web backends.
No it's not. Please cite reputable sources.
One does not need to cite a source of criticism, it is in itself the criticism.
One needs to cite a source for a claim that a particular criticism is widespread.
The burden of proof is on the person making the empirically unfalsifiable claims.
You're playing with words to deflect from the argument. No one has to have proof that they have a criticism. They have to have proof that the criticism is as stated. The evidence is self evident given the OP post.
Many share my opinion, and I don't have to prove that. Nor should I, and neither do you for any position you have. Address the argument.
You have to have proof that a claim is "widespread" if you are asserting it as such without any sources, as you have.
Not only extra packages. They are currently shoehorning extended multiple interpreter support into CPython, which only Django uses. Similarly, the thread support appears to be mainly for the benefit of that famous photo sharing website that uses Django.
CPython is fully commercialized.
As someone that doesn't work for the photo-sharing website, I love the new multiple interpreter stuff (it does feel a bit redundant now that the gilemtomy is going forward though, but who knows, maybe more cool uses will become apparent with time (and even so, it is making cpython-the-implementation clean up some of its ugly c-isms that it never bothered fixing, so it's a net win imo even if the feature itself ends up being not-useful)) and the gil-removal. As soon as the gil is gone I'm going to be experimenting with some new optimization projects at work, whole new flows that just weren't viable before under multiprocessing now suddenly are. (whether it all works out for us at that point is anyone's guess, but it's now our problem and not the languages', so yay!)
And I'm certainly not unique amongst other pythonistas, this has been a goal for literal decades.
Background tasks look well thought out, looking forward to try that.
Despite being somewhat out of fashion, Python and Django continue to serve more down to earth software development well after so many years. I picked it up around 2007 (remember the "magic removal" branch), and it only got better ever since. Many call Python "toy language" or "data scientist language" but the truth is that 99.9% of modern web development is database-backed and, unless you have some strictly computational tasks behind your APIs, the fact that e.g. Rust is 100x "faster" than Python is mostly meaningless. Django just works. Of course, there can be different personal preferences, but I'm really amazed by the project that is closing in on 20 years that is still absolutely relevant and a joy to work with.
I've been looking for know anything resembling django's ORM _including_ migrations in rust, but haven't found a good replacement.
Quart, the ASGI version of Flask, also has background tasks, https://quart.palletsprojects.com/en/latest/how_to_guides/ba..., and there is an extension Quart-Tasks, https://github.com/pgjones/quart-tasks, to run them on a schedule. Via
@tasks.cron("*/5 * * * *") # every 5 minutes
async def infrequent_task():
...
Django getting background tasks built-in is going to be really nice. It sucked having to manage celery or similar alongside it.
You should check out Huey next time. It is a lifesaver for us. Allows us to do the tasks via SQLite db
I use Huey too. Very simple and easy to manage. From a quick look the proposed tasks system seems similar to it.
Yeah that looks really promising. Haven't done Django as my daily driver for a few years now, and I don't recall stumbling upon it back then!
I have been doing Django for most of the last 10 years and I love it. I love Python. I love how well made are most of its core parts. And I love how some 3rd party packages (DRF aka django-rest-framework, django-filter, django-storages) fill the "missing" gaps.
That said, boy does Django must change it's current strategy...
RoR and Laravel have been getting all sort of goodies baked in the framework itself such as tasks queue, nice admin features, integration with frontend technologies (scss, js, even jsx/tsx), even entire packages to manage payment systems, mailing and so on... Good quality features that are ready to use and are integrated inside the framework, which easy to use APIs and fantastic admin integration.
Django devs, on the other hand, have been (imho) ignoring the users (devs) and have been denying any and all attempts at bringing something similar to Django. There are at least half a dozen packages for each "thing" that you want to do, and some are in a very good shape, but others are just hopelessly broken. What's even worse, because they're not part of Django, most of the time there are some edge cases that just can't be implemented without subclassing Django's core.
One of the most recent examples being Django's support for async operations. Except that Django itself doesn't provide a nice way to create / expose an API, which is why people use DRF. But DRF doesn't support async, which is why you need to add "adrf" (https://github.com/em1208/adrf) to DRF. But adrf doesn't support async generic viewsets, so you must add aiodrf (https://github.com/imgVOID/aiodrf) to adrf. But aiodrf doesn't support (...) you get the point. You end up in a situation in which you're pilling packages on top of packages on top of packages just so you can have an asynchronous API. Madness.
Supporting scss it another example of pilling packages on top of packages. It just never ends...
I can't express the desire I have for Django to start integrating packages into its core and get on par with RoR and Laravel.
/rant
DRF is cancer. Try doing anything simple with DRF and you got subclasses over subclasses with overrides and overrides. I hate DRF with all my heart. DRF works only, if you intend to exactly match your model. If deviate just a little bit, the complexity explodes in hour hand.
The biggest gripe I have with Django is that coupling it with modern UIs like Svelte and company is massively complex. The community has failed to deliver a clear guidance on how to handle implementation of such frameworks, including Vite and build steps. Which is the primary thing that drives me away from Django. I'm looking towards Laravel next. Very often, when researching how to implement JS Frameworks into Django I seem to find guidance on how to do this with Laravel and I want to know if it's really that comfy as it sounds.
Love Django, but the issue we’ve had with tasks isn’t with performance, scalability, or devex - it’s with operations. Django-tasks afaict has the same issues as Celery/others in that there’s no Django Admin-like tool that reliably allows you to manage task execution, specifically failures. Without this, I can’t use async tasks for anything that’s important to the business or data consistency generally - which means I don’t use it at all, favoring “normal” queuing backends.
Is Django-tasks the solution that would permit this to be built?
Second this. I’m amazed by the lack of any Django admin interface to start a task or see how it failed, it seems like such an obvious use case. I’ve tried Celery Flower but I found it lacking. I’m surprised because it seems like such a common use case, and given the size of the Django community I would’ve expected something to exist…
Oh man, I wish this was a bit more mature. I am writing a distributed image processing app right now with Celery, Flask, and SQLAlchemy, but I am finding it difficult to KISS. I was tempted to use Django but couldn't justify the overhead. If they get this cleaned up in the next year that would be awesome!
One thing I have learned after nearly a decade in the industry is that you should just never, ever, ever use `celery`. It's overly complicated to program with; the monitoring tools are not great; and the one guarantee you can make is that at some point, you will have some kind of difficult to debug, mysterious failure involving it.
If I needed asynchronous background processing in Python today, I'd definitely be looking into `django-tasks.` If you asked me yesterday, I'd have told you I'd use SQS / whatever the platform equivalent was, or, failing that, I'd probably go with `dramatiq`.
Edit: LOL, I can't believe I forgot to mention what a fscking nightmare `celery` is to deploy and configure. UGH.
Which django-tasks? This one that started development a month ago?
The very same. As mentioned in the article, it's going to be distributed with the newest version of Django.
I don't understand that. The article says '...an existing external project that served as a proof of concept', and the DEP calls it '...an "early-access" demo to get initial feedback'.
Although it looks interesting and I'll keep an eye on it, if I needed something right now, I wouldn't use it.
Does Django-tasks support different workers with different environments? Say I need a worker with GPU capabilities, can it do that?
I don't have any idea. Take a look here: https://github.com/RealOrangeOne/django-tasks
justify the overhead? But you are using flask and sqlalchemy?
You made the wrong choice I would guess. Django does not have 'overhead' to justify in that scenario.
Here's how much is installed when you do `pip install django` currently:
This will probably change when `django-tasks` gets merged in.122370 django python=103934,javascript=18407,xml=29 2734 sqlparse python=2734 1110 asgiref python=1110Here's what installing `flask` + `sqlalchemy` pulls in:
Looks like the same order of magnitude to me, which is surprising. I had always thought of `django` as being heavy weight and opinionated, while `flask` + `sqlalchemy` was light and modular.141314 sqlalchemy python=141314 11464 werkzeug python=11159,javascript=305 8934 greenlet cpp=4993,python=2729,ansic=1092,asm=120 8606 jinja2 python=8606 5842 click python=5842 3907 flask python=3907 2036 top_dir python=2036 644 itsdangerous python=644 487 markupsafe ansic=278,python=209 349 blinker python=349All numbers generated using David A. Wheeler's 'SLOCCount'. :-)
I suspect that people come to the conclusion that Django has performance issues by looking at synthetic benchmark rankings between frameworks. In that sense it may or may not be relatively slow, but it is probably fast-enough for almost all web services in-practice. And if you measure with all of the speediest HTTP interfaces and find out it isn't fast-enough, then frankly Python is probably not the right choice at all.
I've never heard of anybody complaining about Django per se having performance issues. When people say Django is "heavy weight," I tend to think of how it bundles a lot of stuff for you, which can make it easy to build an app; while, at the same time, if you end up deciding you want to step slightly off the garden path and do things juussssst a little differently than Django wants you to, you're probably gonna have a bad time.
OTOH, Flask is too small to impose much of a conceptual framework on you, and SQLAlchemy generally feels like a fairly thin layer of Python syntactic sugar over top of SQL. When I write queries in SQLAlchemy, it seems like they always tend to come out looking more like what I would write in plain SQL than queries in Django do. Granted, both ORMs can start doing things like joining tables behind your back, which can certainly cause performance problems, but that's a problem that's common to both, and not just Django. ¯\_(ツ)_/¯
> if you end up deciding you want to step slightly off the garden path and do things juussssst a little differently than Django wants you to, you're probably gonna have a bad time.
Yep! I am using Flask + SQLAlchemy so I can have an orchestration layer that handles all the business logic, and just takes in my Flask app, Celery app, and database sessions as dependencies. I don't know if there's an equivalent way to do that in Django. If there is, it seems like it would be more trouble than it is worth.
By overhead I didn't mean performance, just all of the extra configuration stuff that comes with Django. Although I'm still open to moving to Django as this project grows. A big benefit of a framework like Django is that everything is already organized for you, I kind of feel like I am re-inventing the wheel a bit here.
I am keeping the business logic separated out though so I can easily move to Django if needed.
The new REPL looks great. Indeed qtconsole on steroid.
Edit: Ooops. IPython magics don't seem to work. I hope they add it.
My real pain point as a user with Django is deployment. Python is so much easier to write, but javascript is so much easier to deploy.
I wish they could make it easy to deploy django or other python framework in a serverless function with decent security baked in.
Where do you feel Django is harder to deploy? I have a different experience and think both are equally deployable.
Background tasks! I am effervescently happy to read this news.
Hah, I just started using Django and was in need of background tasks!