The company behind Stable Diffusion appears to be at risk of going under
futurism.comCalling them "the" company behind Stable Diffusion is pushing it. Just Stable Diffusion the model, but not the idea, maybe.
As far as I know they pretty much just paid for servers to train a big shiny model on that was based on research they had no hand in. Throwing money at researchers after they came up with something good, just to let them build a big shiny version of it, does not retroactively make their accomplishments yours.
Basically they hold no rights to anything relevant, no patents, no secret sauce, nothing. Them going under after exhausting their money will hardly have any effect.
Stable diffusion is just the "brand" of the diffusion generative model, just like Midjourney.
If you look at the stable-diffusion repo, it says that SD is based off a colab with StabilityAI and Runway.
Most of the value from these models is the training + dataset. The architecture is open source, and we've had flavours of it for several years. SD has some improvements on how it handles diffusions, but most architecture out there use about the same, but with wildly different results.
Someone from Runway (Patrick Esser) is actually listed as an author on the paper.[1]
The datasets used are provided by LAION.
Here is a honest summary of Stability.Ai's involvement[2]:
> In their project, the LMU scientists had the support of the start-up Stability.Ai, on whose servers the AI model was trained. “This additional computing power and the extra training examples turned our AI model into one of the most powerful image synthesis algorithms,” says the computer scientist with a smile.
[1]: https://ommer-lab.com/research/latent-diffusion-models/
[2]: https://www.lmu.de/en/newsroom/news-overview/news/revolution...
> As far as I know they pretty much just paid for servers to train a big shiny model on that was based on research they had no hand in.
Which is their business model.
Provide compute to people who can’t afford compute so the only people doing AI research aren’t doing so behind closed doors.
Now, it seems, giving away your product isn’t all that profitable and they need to “pivot” to find a way to keep the business running.
Judging from the interview posted elsewhere in the discussion they make no claim to be inventing anything but just wanted to democratize AI research.
Mentioned in the article.
The article says they aren't the "sole inventor". They aren't an inventor at all.
We’ve had, what, over two years of “basically free” image generation? (Dall-E 1 was Jan 21.) All the major players (OpenAI, Microsoft, Stability, Midjourney) offer free image generation and you only pay for large amounts.
There is a ton of momentum behind the general public’s belief and perception that image generation is free for small uses and cheap for large uses. Most of the other players can afford to keep those losing prices for a long time, too. I think it’s going to be an uphill battle to charge for image generation at amounts that turn a profit. I wouldn’t rule out a more creative way of monetizing it, but the obvious routes look unlikely.
Midjourney has stopped offering free generation. I made all of three or four images and now it is saying they don't have capacity for free trials anymore.
Not having enough capacity is not the same thing as running out of money, though. If the number of users is increasing rapidly (and there's every reason to believe that it is) it could just be that they're not able to spin up new GPUs fast enough to keep up with the increasing demand.
I've noticed that they've recently rolled out some new features (e.g., "/describe") that were at first limited to the higher tiers, but after a short time of evaluating the increased GPU load, were enabled for the lower tiers as well.
To me, that seems to indicate "can't install GPUs fast enough" rather than "running out of money", but of course it's just an indication.
I believe their stated reason for getting rid of the free trial accounts (temporarily, they said) was that people were using bots to register hundreds of sockpuppet accounts to get the freebies. Perhaps they'll come back when and if they can deploy some effective anti-bot measures.
Just a question but is it ever profitable to base a company around an open source product the way SD is? Why would anyone pay money to use the company's model when some guys on Reddit is distributing a similar product, albeit with lower performance, for free, that can be run locally?
What would be the incentive for a person, not a company, to pay Stability AI company instead of downloading and doing a bit of setup to have their own uncensored model?
None of these companies are really directly concerned about profitability. Right now the leaders in ML models are going to be defined by those who have the most capital available to train newer and better models. And the quickest way to get that capital is largely getting investors on board. From that perspective it’s quite rational to release something publicly in a way that is likely to raise the profile of your company. Especially in a world where your main competitors will be training their own models anyways.
Investors make for spectacular customers for a few years. You can sell them dreams! Many companies struggle when they need to pivot away from this model
Patreon and Reddit have figured out the investor grift. First to series-Z is the winner.
What makes you call out those 2 in particular?
Research for the model was done by the CompVis at LMU Munich. Curating the data set was supported by Laion, a German non profit and Eluther, also a non profit.
I do not know the details, but based on the fact that some of the orgs involved in the project literally exists to democratise ai, I believe that many of the stake holders in that project were adamant about open sourcing it.
So the model is mostly a German product? Where are the HN comments about how the EU is behind and that they can't innovate or compete?
In this case they were actually the first to the punch(along with openAI)!
The nuanced version I've heard in other tech industries (bio/agri-tech) is this: a) Europe organisations and gov't does all the deep research; b) US private companies turns it into products and reap the PR and money.
In fact, I know of a huge company that does all their research in the EU, where they can get gov't funding, but then the process to turn them into products is in the US (less regulation, larger market).
Other comments highlighted some aspects, but one thing that should not be underestimated is that German hacker culture has strong anarchistic tendencies and there just is no place to be creative in cs and make money, at least not in the same way as in the us.
So lots of highly talented people just do open source in their free time or work at universities.
Here are some interesting bits on deep learning vs crediting prior research from a prominent Swiss DL research pioneer: https://people.idsia.ch/~juergen/scientific-integrity-turing...
> As mentioned earlier,[MIR](Sec. 21) when only consulting surveys from the Anglosphere, it is not always clear[DLC] that Deep Learning was first conceived outside of it. It started in 1965 in the Ukraine (back then the USSR) with the first nets of arbitrary depth that really learned.[DEEP1-2][R8] Soon afterwards, multilayer perceptrons learned internal representations through stochastic gradient descent in Japan.[GD1-2a] A few years later, modern backpropagation was published in Finland (1970).[BP1] The basic deep convolutional NN architecture (now widely used) was invented in the 1970s in Japan[CNN1] where NNs with convolutions were later (1987) also combined with "weight sharing" and backpropagation.[CNN1a]
It's pretty amusing how a lot of people that really don't like the EU and speak against it here are not from EU.
I think there's a lot of "open core" stuff like Sentry, Tailscale (I think?), or Gitlab. Where "enough" of the house is given away in theory for you to do it yourself (but why would you do that?)
Metabase is another company I knew where they do this, but I don't know how successful they are (their hosting costs start at "way too high considering how easy the thing is to spin up", which was great for me at $PREV_JOB I suppose but)
Perhaps the business case is for "we want the Open Source product, but we're in a compliance-oriented business and need to have someone selling us a support and maintenance contract behind it."
I'd suspect the moment you touch anything with external auditing, it looks better to say "we've got a fully paid up support contract from the vendor" than "we're running v1.23.456-ubuntu-patch-357-with-chives-and-salsa that we downloaded last week."
I'm concerned about Metabase because of this. Amazing project, but the cloud offering doesn't seem to be enough.
> Sentry, Tailscale, or Gitlab
None of which are profitable
We should outlaw government spending on closed source, non public domain software. That would create a huge market for open source companies to thrive. Would be a win win for everyone (except the current closed vendors who provide government with crap software and get sucker taxpayers to foot the bill).
The issue is not "Microsoft Office vs Libre office" but rather "how much does it cost to necessary up to the support necessary for the government office in order to support Libre office?"
Things like "ok, we can't use one drive - need a different tool," "can't use Sharepoint, need to use a different tool," and so on.
The market is there. RHEL and similar are well established.
In order to make this a "we should do this" either government IT funding needs to be significantly increased (that is difficult in the current political climate in most places) or the support offerings and staff needed for the average user (using Windows, Sharepoint, Word, Excel, Teams, and Project) needs to be competitive with the pricing that Microsoft offers.
That should be "simple" - make a company that offers the same level of support as Microsoft does for a packaged suite of software that includes easy installations, appropriately locked down desktops, call center, and so on.
And if that can't be found at the same price that Microsoft offers - then we return to the "increase government funding."
Saying "we should outlaw government spending on closed source" misses a lot of the tools out there that are needed to keep things running. Is there a FOSS (with support contracts for the stack) alternative to Cerner or Epic? SAP? ArcGIS? And that's not even getting deep at all into the niche SaaS tools that some pieces use for specific problems.
The market is there and state and local governments would likely jump at the opportunity to switch if there's a company that can offer the same functional stack with the same support for the same price or cheaper built on top of FOSS. Otherwise... persuade those state and local governments to staff up to the necessary levels to be able to hire people able to customize and support the FOSS to fit their needs and be prepared to financially support that decision.
> Cerner or Epic? SAP? ArcGIS?
Of those only ArcGIS isn't a basic CRUD, and yes, there are plenty of open source replacements for it. It's in fact way behind the open source tools.
There aren't for the others, because they are about paying a company so you can use its other customers data, or paying it to get overpriced consultants. None of them are about software.
Epic/Cerner/SAP may be CRUD apps with clunky and outdated interfaces, but I wouldn't classify them as basic. Those applications are massive and have a ton of legacy code to connect different things. I'd suspect much of the rest of their staying power does lie in the company's scale and relationships, as well as their domain knowledge. It would cost massive amounts to build a viable competitor to any of them, and you'd need people who understood the problem space well enough to know what code to write. Doesn't matter if it's just a basic CRUD app built on top of a standard RDBMS, if you don't know what features are needed.
I agree, but realistically that’s tough. Sometimes the government just needs a tool like everyone else. And something that works and is available today off the shelf is the best you’ll get.
Sure it’d be great for government acquisitions to subsidize open source, but at what cost?
Perhaps a "you can pay for closed source but must prove you looked into using open source first" kind of thing? Much like the restraints on employment immigration so that companies must try and employ locally first.
Hypothetically, say an organization is looking for bug tracking and project management software that is cloud hosted.
This is for 200 users and it needs 24/7 support. The budget is $30k/y.
Ahh, you could spin up Redmine on AWS for much less...
Maybe, but I'm going to need to hire a Redmine consultant to help configure it for our needs, add maintaining and updates for that instance to the sysadmin and add it to the helpdesk support - and if we really want 24/7 support, our helpdesk is only 7-5 business hours (the support is for the 'it crashed' which means that the sysadmin gets it on pager duty... but if there's an issue that's application support, so wake up an application team person who isn't on pager duty...). And there's a bug in Redmine, so either get a developer to learn Ruby (this is a Java shop) or beg the core team to fix it...
You know, Jira is looking more attractive as these likely costs start adding up - and it's a fixed price... not going to have an AWSh.. surprise if a bitcoin miner exploits an issue on the base image that didn't get updated.
Local and state governments do look at open source first. The "no cost to start" is very attractive until they get to a "this is a hard problem, you're on your own" and the costs go way past what a commercial application costs.
For government, a predictable budget so that you can ask for next year is valuable. Unknown budgetary costs are were overruns happen. A budget line item of $26,500 per year is much better than a budget line of "$0 to $50,000."
RedHat is the exception that proves the rule: no. And even in RedHat’s case, the profits came from services, not software.
Databricks is a pretty good example of a huge company built around and open source project (Spark).
Was it profitable for Cygnus? Red Hat? Android? MySQL?
> Was it profitable for Cygnus?
We were a bootstrap so were immediately profitable.
I only see something like SD to be attractive for personal uses, not corporate. That is why I mentioned a person, not a company, as a customer.
If your product is tested and guaranteed to certain standards, like MySQL and RedHat, then that is something a company may pay for. But a user doesn't really have that high standard so they can be satisfied with just the off brand, derived stuff floating around.
There was that post maybe two weeks ago about the blender artist who was afraid they were out of a job because apparently some ai can do his sort of work based on prompts. Stable diffusion most definitely has use in creative industries, at the very least, being able to render a representative first draft that actual creative humans can work off of.
Not MariaDB
> around an open source product the way SD is
The Stable Diffusion models are proprietary freeware, not open source.
I wonder if, then, Stability AI wanted to sell license exceptions—namely, the ability to use the software for amoral and (mildly) immoral uses.
A closed source NSFW model that they sell API access to... PR and legal nightmare... could be very lucrative
I pay for stability.ai because the API works and is very inexpensive.
It would be a huge disaster to lose out on the folks releasing the models for free.
Nobody benefits from their failure.
If Runway and Stability can cut costs they will become cherished institutions.
> cut costs
without a stable source of revenue, cutting costs mean nothing. What are they selling, and why isn't there buyers?
A business cannot survive without revenue, and revenue only comes from people who want to buy something from you that they cannot otherwise get else where.
I applaud StabilityAI for releasing SD for free. They could've done what openAI did, and monetized it (which other diffusion model services are doing). By releasing it for free, stabilityAI contributed to the common good. Unfortunately, if there's nothing else that can be sold as a product, they cannot be sustainable.
An alternative, which i'm not too big a fan of, is collectivization of AI models, and make it a global commons for which taxpayers will fund.
They could have at least charged money for it. "I'll give you this 4 GiB model file for $50" might have been a better business model than "lemme give this to you for free!".
> releasing the models for free
...
> cut costs
Unless they can cut costs to zero, something has to change.
It seems like mandating just a tiny bit of usage data back to the model would give SD a massive lead on training data, but I'm not an expert. Maybe that's happening already.
Like, example. I use SD in Blender sometimes as part of the compositor. I have maybe a 10% acceptance rate for SD output: sometimes the water isn't right, or the clouds look goofy, or something keeps getting rendered as an anime pillow for some godforsaken reason. If SD captured my prompt history and some of the final model tweaks between runs, they could ostensibly get really solid HITL test data. Then they could be the curator of that "super model" which they could upsell, maybe along with very high rez stuff, or a higher priority on jobs. Again, not an expert, so who knows. And also, having the model local, that gives you back some of the same benefits, but without the scale.
Interview 7mo ago w/CEO of StabilityAI.
Not too surprised about funding issues from the casual answer.
I’m not saying it was bad to self fund a project, but having to choosing between your life and fun (and potentially very profitable) projects is not easy.
Note that stability have been funding freelance researcher by providing compute resource such as RWKV[1], Open Assistant, some works by LAION[2] and lucidrains[3]
[1] https://github.com/BlinkDL/RWKV-LM
[2] https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K
[3] https://github.com/lucidrains/gigagan-pytorch#appreciation
The means of production are cloud GPUs, the winners will be those who own them.
In a gold rush, sell shovels. And Nvidia's stock price bump reflects that.
This time around though, the means of production are available to anybody with a credit card, and AWS/GCP/Azure throw credits at startups that apply, just to have them locked into their cloud. Break free of the chains of work, with generative pretrained transformers!
Stable Diffusion was obviously never intended to make money, but as a Napster-style maneuver to enclose all image data on the internet through AI picture generation. They release something to the public that would get sued into oblivion were it done by an established company, but oops, the genie is out of the bottle, what you gonna do, the only choice now is to restrict it to a few power players who can actually do the necessary rights management. Artists and photographers will get the same shit deal musicians did in the streaming era, i.e. an algorithmically determined pittance for use of their work. Don’t like it? Oh well, either settle for that or people will fine-tune a pirate model on your work in half an hour!
To put it politely, your comment is bullshit. Stability AI is a billion dollar company with hundreds of millions in funding.
Not saying I agree with the GP, but I don't understand how your comment refutes anything? Their valuation and funding doesn't necessarily mean anything.
In fact, I think one could argue that many companies raising that much money early on are indeed trying to become a first mover and don't care about making money (yet).
I would think the last decade or so of the tech industry would demonstrate otherwise? You could say the same about myriad other startups doing legally questionable things until regulators caught up with them or interest rates got raised. That someone throws piles of money at you and says your worth billions does not make that reality.
Motte and Bailey.
is it “company was planned vehicle for Napster except freeing images not music”
Or is it “an instance of companies wasting money because ZIRP and some companies are bad like some stuff I disagree with that happened at Uber”
Those two things aren’t mutually exclusive. Uber pursued both of these strategies (and lost massive amounts of money while breaking laws), but with a net benefit to their investors by weakening labor protections and other regulations.
100 million, actually. and Semafor seems to be asserting that most of it has been spent in the last 6 months since they raised it.
It’s going to be so much easier to push these models out to the edge if they are open source (see ggml, diffusion bee etc).
This won’t be possible without accelerator compute for training. Open source developers can’t afford this compute.
Apple benefits because they can deploy on iOS devices. Amazon benefits because they can tarball the models and sell it as a managed service.
I don’t think either wants to be in the business of figuring out which researchers is going to use the compute for deep fakes or for the next big model.
So something like Stability should exist, Amazon and Apple should figure out how to make that happen.
We're going to see a lot more of this. Companies using practically free debt to 'grow' fast with no actual real path to profitability, suddenly finding it hard to repay debt and attract new 'rounds.'
There's a heap of bad banks and terrible debt floating around, the golden goose has been cooked. Turns out risks aren't just things you can ignore for 'growth.'
Very unsurprising, as I said before on unprofitable AI companies doing it all themselves and running out of money quicker. [0] They will probably run back to VCs again to attempt to raise money at a lower valuation.
Focusing on hype and growth over profitability and burning hundreds of millions of VC cash on data scientists and AWS for training, fine-tuning AI models. This is even before mentioning the mounting lawsuits they are already facing.
The inefficient training of deep learning models is unsustainable for these pre-profit companies.
It’s rough because there are lots of models that can do a decent job competing
I mean just let it die, it's the investors money that sinks
That’s why they need to push UBI harder. They’ll just do it in private as always have been and we will lose everything overnight. Next day you useless human beings in gulag or unit 731.