US Cybercom says mass exploitation of Atlassian Confluence vulnerability ongoing
zdnet.comThe good thing about the fact that Atlassian offers both on-prem and cloud versions of their offerings is, everyone is now aware of the awful engineering practices that underpin their products. We have to assume that there are problems of a similar nature in their cloud service, which is way more of a problem considering the number of orgs that depend on the JIRA SaaS offering.
Maybe the founders could have used some of that time spent planning a tunnel between their side-by-side $100M houses, or engaged in Twitter rants, to actually bother delivering value to customers. It’s only a matter of time before this product suite is disrupted, and it might represent one of the most obvious low-hanging opportunities in our entire industry.
I still remember being in line at a WWDC a few years back, overhearing someone ask a developer, “where do you work?” When the developer responded with “HipChat,” the other person immediately chuckled and said, “oh — Atlassian... I’m sorry” — and then everyone around them also started laughing. It’s amazing that this company continues to fall up, and that the founders have taken on roles as the ruling digital gurus of Australia (shows you why it’s so easy for the government to run circles around the local tech industry and pass whatever laws they want).
> The good thing about the fact that Atlassian offers both on-prem and cloud versions of their offerings is, everyone is now aware of the awful engineering practices that underpin their products.
Regardless of what one thinks about Atlassian, this is a completely ridiculous bullshit statement, and anyone who works in the world of business software knows it.
I don't think there is a company out there that hasn't had critical CVEs, nor most major open source projects, either.
Microsoft had a recent vulnerability in their Azure Cosmos DB product that left thousands of customers' data unprotected. Google has released multiple patches to Chrome in the past month.
If you demand you'll only use products from companies or open source projects that have never had a major CVE, you'll be writing a lot of your own software that probably has even worse security.
You are missing the point entirely.
Any sufficiently complicated product will eventually have major CVEs, as you say. Anyone having hosted Atlassians product know that these products are nothing but garbage fires on the inside, as the commenter above said.
Both of these statements are true and not mutually exclusive in any way.
However, one does not conclude from the other as is insinuated in the comment.
If all products have CVEs, and CVEs are a form of defect, then it follows quite naturally that highly defective products will have CVEs, likely more than less defective products.
So yes. Yes it does. Unless you meant that CVEs imply garbage code, in which case I think you read the comment wrong.
But here we're talking about one particular CVE:
> everyone is now aware of the awful engineering practices that underpin their products
This one fault doesn't tell anything about overall quality of the product.
This is a complete misunderstanding caused by not readinf the first half of the first sentence of the comment.
The full sentence:
> The good thing about the fact that Atlassian offers both on-prem and cloud versions of their offerings is, everyone is now aware of the awful engineering practices that underpin their products.
I.e., because the software is available for on-prem deployment, we have experience deploying, managing and debugging it, and therefore know that it is garbage, and we can apply this knowledge to conclude that their cloud offering is garbage.
It does not say the product is garbage because of a CVE.
It follows in theory but may not in practice. Certain software may have lots of defects that are not CVEs.
All products will have defects that are not CVEs, hopefully by a large margin.
But every sufficiently complicated product will have defects of this type, and a higher probability of defects does increase the probability of CVEs.
So, it follows in practice, as can be seen by looking at CVE rates from projects with high defect rates (web browsers, kernels, ...) vs projects with low defect rate.
Note that defect rate differences can be caused by multiple things.
> these products are nothing but garbage fires
Would you care to give us alternatives, for example, to the JIRA bug tracker (which I used a lot, slowly :-))
> Would you care to give us alternatives, for example, to the JIRA bug tracker
Paper.
An Excel spreadsheet shared on a Windows for Workgroups share drive.
Carrier pigeon.
Quitting software to become a llama herder.
Seriously, though, after over two decades of using different tracking systems I think that the real alternative to stuff like Jira is to not go there. You may think that you need all those bells and whistles, but you really don't. What you actually need is the simplest possible issue tracking system you can get away with. All you really need is a prioritised list of issues, and a list of who is working on which issues now. ‘Kanban’ boards get pretty close.
Minimalism is power.
I don't disagree that Atlassian programs are generally bloated, but it is interesting to see everyone jumping at JIRA when its Confluence with the CVE...
JIRA is too many things to too many people - in the quest to be everyone's bug tracker, they wrote (badly) in the whole kitchen sink. This is a general issue with most software, though. Especially big software.
You cannot have complex capable software that is also simple. That's an oxymoron. You can have complex software which works, but it is difficult to get there, and keep it there. There is simply too much that can go wrong that over time, it almost must go wrong.
Keeping everything 'simple' is not an alternative - the world is complex and so even a bunch of simple things, when put together, make something complex. Think unix shell scripts which can be so much sphagetti. The unix philosophy is to keep things simple, and yet ignoring the complexity leads to its own complexity.
That's the real 'terrible engineeing practices' which get down Atlassian and everyone else's programs.
Someone said 'use paper' or 'quit and become a llamma herder' - these seem simple but again, paper burns, shepherds get shot or held up by drug gangs.
All to say, I don't think JIRA is actually that bad. There could be worse products, there could be better. But it would exceedingly difficult to make something simple which also served everybody.
I would prefer to use anything but JIRA,I fucking loathe it.
Jetbrains Youtrack is soooo much better in any and all ways that I can think of. https://www.jetbrains.com/youtrack/
ZenHub! I love, as an engineer, that it layers on top of GitHub issues. So, in my general day to day I never leave GitHub.com or GHE, while project managers get all their shiny toys in ZenHub that plays nicely with GitHub issues.
Pivotal Tracker. I've found it to be a much more productive tool, and teams using it typically move alot faster and are far happier than those using Jira.
Is TFS no longer considered a competitor? Feels like it should have been the first mentioned here. Not saying TFS is problem-free, of course.
I've had to use Azure DevOps recently, and it made me long for the speed, simplicity, and stability of Jira...
The most annoying bug was that one could only have around 4-5 tabs open before something went wrong and everything stopped updating.
Azure DevOps?
Spolsky’s FogBugz is still out there after a few ownership changes, and is still a hell of a lot less painful to use than Jira.
It was innovative 13 years ago, but never really caught on.
Kiln was also a great product and allowed for using both Git and Mercurial. It was way better than anything else at the time, but lost out to Github.
I always liked Spolsky's Evidence Based Scheduling that was built into the products.
> Kiln was also a great product and allowed for using both Git and Mercurial. It was way better than anything else at the time, but lost out to Github.
I used both FogBugz and Kiln at a previous employer, and I really liked FogBugz, but we had nothing but problems with Kiln. We had a fairly large team (maybe 75-100 people) and a good sized project (maybe 300k lines), and it was painfully slow to do anything in Kiln and it was down/broken at least once a month. It wasn't so bad early on when we were small, but it didn't scale with us very well.
No. It isn't. Having recently moved (a year ago) from FogBugz to Jira it's like a fresh breeze has blown into our case management process.
Where may I learn more about exactly how they are "garbage fires on the inside"?
Thanks
Atlaskit (their UI framework) is open source.
It is very enlightening. Try using their editor component to display the text of one of your Jira comments and making it display attachments.
find or create an installation and examine how the database is put together
Try to use one
Try using two of them connected together.
In order to do that, one would have to try to connect one to the other, which is usually sufficient.
First you need a fire starter, which in this case must be made out of bills valued 100 USD each or greater. It'll take quite a few to light the Datacenter licenses that are the now the only on-prem tier on fire...
So... you do not, in fact, have an answer to the question?
Lacking humor I see.
My point is that it is experience acquired from managing these for a decade. If you wish to acquire this experience you need to run it, which is now even costlier than it used to be.
Actually using these products suffices. Especially confluence, which regularly reduces me to a stream of particularly foul profanity.
I don’t disagree with what you’re saying about software having ongoing vulnerability issues. But, that’s exactly what the problem is with communications-centric solutions that don’t offer strong data security protections such as end-to-end encryption: you’re always one CVE away from having your company’s data exposed. And, in this specific case, there is a MAJOR difference between Chrome getting owned, and the program that hosts all of a company’s internal communications concerning, among other things, known vulnerabilities in their software.
Think about that for a second. If someone finds a vulnerability in JIRA, they don’t just find a vulnerability in that software: they’ve got access to support tickets, issue tracking, etc about lots of vulnerabilities in lots of software. That’s a big deal.
The fact that the US government had to step in and say PLEASE TAKE THIS SERIOUSLY, rather than Atlassian going into a Code Red situation, shows that they just don’t take the level of responsibility they’ve been given as seriously as is required for what they’re doing. This isn’t just some lousy app having a CVE. This is the keys to the kingdom for a lot of very critical software. This is systemic risk. The problem isn’t the code, it’s the culture.
If you “work in the world of business software” and you think that’s a “complete bullshit statement,” I really hope you don’t work on anything for which such systemic risk is possible. Because, to turn your statement back on you, that’s a complete bullshit way to treat the responsibility you have for the data with which you’ve been trusted. Go build a social media app or an online shopping site or something, and stay out of critical systems that can create cascading vulnerabilities.
For people who have the Atlassian Cloud offering this is not a problem and has been fixed. The only people who are impacted are the one who host Confluence themselves. There isn't much that Atlassian can do except that tell them to update the software.
If you are maintaining the software of someone else and this software is exposed to the internet that's your responsibility to update the software. If you want your service to be available from the web and have good security, use the cloud offering, that's what it's for.
And if you don't want to use external software to manage your internal knowledge, then create some shared windows folders that nobody use and quickly becomes a mess. What alternative do your propose ?
> and stay out of critical systems that can create cascading vulnerabilities
Oh, you mean like Microsoft and Google in the examples I gave?
Why are you changing the goal posts? Your original statement was that this issue is evidence that "everyone is now aware of the awful engineering practices that underpin their products." My clear argument is that this is not the case, as lots of companies with systemically critical software also have bugs of this magnitude, or more.
Much as I hate everything Atlassian touches (although Portfolio was pretty useful as a stand-alone tool if you kept it far away from JIRA), It’s not like MS Office or Sharepoint have never had vulnerabilities..
> don’t offer strong data security protections such as end-to-end encryption
Imagine being so naive as to think that documentation would be improved by e2e encryption. It's not bad enough convincing developers to write things down, now we need to explicitly share those things with every new person who joins the team?
"Sorry, we can't fix your bug for 6 weeks, Bob's on paternity and the fix is documented on his page".
> MAJOR difference between Chrome getting owned, and the program that hosts all of a company’s internal communications
There's at best 0 difference between these things. Pop chrome, harvest tokens, access Jira. Think about that for a second. What critical company information is not accessible to someone who has arbitrary code execution in your browser.
Oh, serendipitously, the article currently directly below this one on HN is "Apple iMessage Zero-Click Hacks". So add Apple to the list of companies on your "awful engineering practices" list.
Eh? If there was a remote root exploit in Chrome, ALL your data is similarly 0wned, exfiltrated, and for sale to your enemies. EVERY computer you have used Chrome on has is now suspect, and all website data each of those compuers to will now have its data stolen and sold on hackers.ru and .ch. All my employer's business data being stolen is one thing, but all of my online banking credentials are of particular personal interest to me in staying confidential.
> awful engineering practices that underpin their products
Confluence has become an absolutely disgusting, bloated Javascript beast. The amount of JS that it loads is unbelievable.
Atlassian has a reputation for poor engineering/backend practices. It's a great (to use) product though.
Interestingly I assumed (due to personal and anecdotal experience fro colleagues) it’s not a good experience and that was part of what the parent was referring to.
Coming from ClearQuest, Jira was a breathe of fresh air some years ago. We can run projects with several hundred to thousand people with it (the on prem version) without problems and UX is ok IMHO. Maybe it's easy to screw up the config?
It's slow, but so long as you don't do anything “out of the ordinary” (i.e. try to act like the average sort of person who would use Jira), it's decent enough to use. I've personally had no workflow issues, though I could tell there was something deeply wrong with the internals.
Then again, if you're only using the “ordinary” features, Jira doesn't have much advantage over any other bug tracker; Gitea can integrate with Jira just as well as with any other bug tracker, and well enough that the Jira / Confluence integration isn't necessary.
The Atlassian softwares are okay, but (from my limited experience) worse than their alternatives. On several occasions, I expected a bug, but it refreshed or redirected, and there wasn't a bug. I have found no bugs while using Jira, and not unusably many while using Confluence… but I can say the same for Gitea, GitHub, Gitlab and even Bugzilla. (Gitea's native issue tracker is actually good enough – and therefore better than Jira – for everything I ever used Jira for.)
> Jira doesn't have much advantage over any other bug tracker
Ohhh found the non power Jira user! Gitea is good enough for your org. Without knowing your requirements this is pretty much a blanket statement. Gitea is nowhere near good enough for a large engineering org.
Example: groovy integration with Jira allows an enterprise to build their own custom workflows. Like integrating with your Vcs to enforce funded work (requiring an open Jira issue in git commits). Features like this, until recently, left them in a league of their own. GitHub Actions has leveled that playing field. Gitlab and gitea can fight it out with bugzilla, but you won’t find them in most engineering orgs.
Another example: building out complex dev roadmaps ahead of time with constraints and relationships. Bridging the gap between dev, product, and project management
> Not trying to dig on your usage of issue tracking
> but you won’t find them in any worthwile engineering org.
sure.
Examples?
It's an ok product until you run into performance issues. Which due to said horrible backend engineering is numerous. Also their support is awful.
Don’t worry, the datacenter edition takes only 200ms to load their HTML, and then another 15s to generate their JS package on every single page load.
I don't remember a company like whatsapp having this kind of problems.
(note that this is not a complete list) CVE-2021-24026 CVE-2021-24027 CVE-2020-1891 CVE-2020-1909 CVE-2019-11933
Perhaps you should consider that not knowing a thing is very, very different than that thing not being true...
Just FYI, the grammatically correct phrase would be "these kinds of problems" when plural versus "this kind of problem" when singular.
I can see how you jumped to the conclusion this CVE means Atlassian is nonsense, but it’s not the only take on the comment. The discussion arising from a
I’m not really sure what the point of the rant is. It’s not as if such a comment conclusion is as big of deal to reality as an idiot staying unvaccinated.
But I get it; someone is “wrong”* on the internet.
* where wrong is defined very specifically to one or a handful of particular readers but the error doesn’t rise to being a real problem for humanity
Downloading 20MB of javascript to view a wiki page is all I needed to know that Atlassian is a garbage fire of acquired products stitched together.
Well that and spending any amount of time using it and feeling the crustiness.
JIRA has a solid use case (if maybe breeding ground for CVEs), confluence is just a bad competitor to OneNote. Its not a wiki...it stopped being a wiki a long time ago.
Mixed frontend and backend rendering = horrific slowness and impact on productivity
There are many jira alternatives out there, from what I can tell. Why are they not disrupted already, if it’s such a low hanging fruit? (Honest question - I don’t have any personal preference)
Atlassian products are vast, integrated, and support all the crazy draconian processes that every insane project manager wants to implement.
You can't easily dump Jira if you are using Jira, confluence, bitbucket, and whatever their CI/CD product is called (bamboo?)
Not just product managers. All kinds of enterprise processes decompose nicely into tickets. Arbitrary workflows, transitions, validations, custom fields, filters, and reports make it one of the most successful “no code” tools behind Excel, and then the REST API is comprehensive and well documented. At my company JIRA bots are compelling and cheap alternatives to new web UIs for internal tooling needs, surprisingly often.
How many PMs actually use those features? In my organization, for example, I don't see any reason why we should prefer Atlassian over Taiga, other than familiarity and inertia.
There's a ton of things you can do in Jira, and different organizations use different things. It's like excel or word, in that sense.
> How many PMs actually use those features?
It isn't the set of features that makes them sticky. It is the 10% of features the lead PM can't live without, and the slightly-overlapping-but-different 10% of features the team PM's can't live without, and the slightly-overlapping-but-different 10% of features the developers can't live without, and the roll-up report features the leadership team can't live without... There has been some research into this phenomena [1]. I'm not aware of a handy term for the phenomena, would surely appreciate someone pointing me to it, and the industry recognizing it and building for it, though.
Internally in my head I call it "multiplexing feature sets". Though I never say that aloud and just give the long-winded explanation if I have to bring it up in meetings.
> ...I don't see any reason why we should prefer Atlassian over Taiga, other than familiarity and inertia.
In large-organization dynamics, never underestimate the familiarity and inertia factors. I regularly see vendors screw this up so badly. Account management teams of even large organization vendors (so they should be taking action upon this, but apparently are not) regularly have no clue what kind of massive red flag it takes to push their decade(s)-entrenched product from "man we wish this could be better" to "you really need to fix this, or we need to start looking elsewhere".
How do you tell when a customer is just kvetching and presenting empty threats, or is laying it on the line to you? Easy: the customer is happy to share with you specific details of how the product shortcomings/defects are impacting line of business processes and the business impact (ask for this under the cover of learning what the critical business impact is so your support teams can devise the most appropriate tactical technical solution for immediate partial/total relief), and is eager for live working sessions with the customer's engineers.
And understand this: rarely will a customer reveal they have switched away until it is practically finished. They want to maintain what little interim support quality they have left, until they can step off. If your account management practices engage "all hands on deck" priority support when you get wind of a competitor in the wheelhouse, then you've lost before you even started. That's why the reveal is always a shock to vendors, and always a done deal when it is shared.
The time to understand that your product and product support consumption experience for a particular set of customer needs is horribly broken is when you notice a customer requesting product support leadership engagement more than 2-3 times in a year. If you aren't tracking those engagement touch points, then I guarantee you are losing license sales. They just don't show up in the sales metrics.
[1] https://www.sciencedirect.com/science/article/pii/S187770581...
What are these features that PMs can't live without, that they don't get with an alternative such as, say, Pivotal Tracker?
I think that the larger and more dystopian organisations cannot do better than Jira to fulfill their needs.
GitLab has wikis, CI/CD and sprint / issue / project management features. We use it at $DAY_JOB and while not perfect it's a pretty great offering.
Clubhouse (soon to be renamed Shortcut) covers the first two. Github covers the latter two. It's easier to switch than ever.
No it's not easy to switch. Engineeting Organisations have invested a lot in the Jira eco system, that includes custom workflows that are understood only by a few to be able to alter them, users are productive right now with jira and nobody wants them less productive even just for a while learning another task manager. Here again the integration effort to migrate would scare any leader who would be blame for the impact on so many teams deliverables. The effort is enormous and error prone to take all the data in there, move it elsewhere, rebuild the workflow and integration configuration, while keeping track of lack of feature parity to write down an explanation before everyone complains and pre empting their request to at least provide a work around what they used to be able to do with 3 clicks. Confluence slips right in because it integrates very well with jira. Just embed a confluence page into a task, and just mention a jira task within a confluence page and you get a seamless experience. Yes we could use another wiki , and a good one as confluence is a calamity. But convenience is what organisations are after. Perfect is the enemy of the good they will say. I don't have a love for atlassian products, but they tend to do their job very well compared to the majority of the competition. You will always find one product that compares, or even is superior but overall, their product works. So here we are.
If they get plagued by further security vulnerabilities then companies handling sensitive data will concede to migrate, but it won't be simple by any means.
If you believe changing people's habits is easier than ever, it's rather the opposite. Workers are less and less inclined to learn any other way . The alternative has to be order of magnitude better than what they are using, otherwise they will resist the change. The fact is atlassian provide good to great products overall.
> "Just embed a confluence page into a task"
Except that confluence and JIRA are both so slow that you'll still be there 2 minutes later waiting for it to load. Perhaps not everybody's experience is so bad? I don't understand how anyone could consider these products convenient given how ridiculously slow they were.
We used to do refinement meetings over video call (I guess everyone is these days) inputting into JIRA, and we'd literally spend 3/4's of the call waiting for JIRA to catch up with the words I'd typed. There's bad engineering, and then there's making writing a sentence text box lag with times measures in seconds.
We use the cloud version, I've edited huge Confluence pages and it's typically very snappy. Jira likewise. Could be an under-resourced on-premise deployment?
I think they’re talking about the front-end, not the server. FWIW both Confluence and Jira are abysmally slow on my machine (new MBP) on the cloud version.
I use both Jira and Confluence (cloud version) on a 6 year old Dell laptop and have no performance issues.
Maybe I'm closer to their servers. Or you're using Safari.
Nope, I’m using Chrome / Firefox and it’s a common enough problem that Atlassian sluggishness has been a running joke on multiple teams I’ve been on.
Maybe I should measure it and post somewhere, I thought it was a well-known problem but maybe there’s something different about our setups.
I have seen on premise issues , infrastructure can be the problem, or config. Jira cloud can also be a problem,but nothing one can do about it other than sending a ticket. I think they know what they are doing wrong with certain "low value" subscriptions.
Is Safari slow when using Jira?
I use the cloud version also. While Jira doesn't have any issues, editing Confluence pages can be a right pain. It is constantly "losing my connection" and prevents editing until it's happy again. There's no connection issues present, unless they're at Atlassian's end.
Watching the network tab, all I see is a requests to `bayeux-sync1` endpoint that take 5 seconds or longer. There's always one of these requests hung when it enters this unresponsive mode. I suspect this is some part of the syncing system for multiple editors. Which, if true, makes it particularly hilarious to me that the entire point of this system would be to sync multiple edits, yet it won't let me keep editing while it's waiting...
> Clubhouse (soon to be renamed Shortcut) covers the first two.
Even taking the following into account?
>> Atlassian products are vast, integrated, and support all the crazy draconian processes that every insane project manager wants to implement.
Well if your organisation wants crappy project management tools and processes then there's nothing to be done. But there are plenty of alternatives out there for those who seek them.
There aren’t good integrated replacements. Any system that lacks a wysisyg interface for text is not a viable option. If copy and paste from Word doesn’t keep some of the formatting, forget it.
Our support board has customized forms. These forms create tickets that can seamlessly be moved to our scrum board once they are vetted. We use JQL to manage a lot of the boards we use. We have custom workflows for different ticket types. We use the comprehensive access controls to grant partial access to users based on custom roles.
None of this rigidly enforces our workflows (aside form access control). Instead, it streamlines sharing information across departments.
When it comes to a company wiki system, Confluence is extremely hard to beat. (I’ve use so many wiki systems. Dokuwiki is my goto.)
All of these systems use a single user account. We use single sign-on, but we don’t have a large IT department to manage the dozens of services we access every day.
> There aren’t good integrated replacements
I think even that would be a good attack vector against Atlassian: For them, "integration" means adding links from one product to the other. If I were to pay for an integrated suite of tools, the least I would expect is that their bloody markup languages are consistent. But because Atlassian just buys random products and then doesn't seem to ever change a single thing about them, that's not going to happen.
That’s false: Confluence went from Wikimarkup in 2008 to wysiwyg in 2010 (to big uproars); and they are uniformizing all their cloud products under the new ADF editor.
Thanks, that's good to hear. A quick Google search makes it sound like it will finally be possible to use `backticks` for code everywhere.
It is actually possible to want bespoke tools to do certain things. And honestly? Confluence is slow, but at least it has all the features I could want and it works.
People are like”use the shiny thing” forgetting the existing thing has, you know, stuff I actually use.
Thanks, I got you. But you didn’t really address the point of the person you responded to, then, which was that part of why this space hasn’t been disrupted to a larger degree is that Atlassian products are entrenched in many companies, and that a lot of people do want that complexity (disagree with that as you or I might).
> ... and that a lot of people do want that complexity (disagree with that as you or I might).
As much as I love to hate Atlassian, I feel like complaining about Atlassian is like techies complaining about management in general. Every single anecdote is both terrible and true, but it's not quite as easy as it seems to do it better.
This is the primary failing of many tech people/companies. You can't compete by just building a "better" technical product.
Everything from sales to support to customizations and integrations matter to companies, especially as they grow and develop their own teams and management structures which require the software mold into their workflows. Whether the management processes are the best is a different conversation, but being able to support any scenario is why Jira and Confluence are so successful.
It's the same reason why Salesforce has dominated CRMs even with so many "modern" alternatives around.
> being able to support any scenario is why Jira and Confluence are so successful
It’s incidentally also exactly why they’re often horrid to work with.
Because Jira is flexible and has the needed features and integrates with most things you care about. To the point where everyone in the org can tolerate it. Alternatives tend to focus on one group of users and the rest HATE the product.
I've tried a lot of these products and in the end come back to Jira because it works better on average for everyone.
This comment nails it - anyone who has had to investigate other offerings that satisfy the needs of project management, design, and engineering will quickly find that all the other options out there are total garbage in comparison to Jira, which is why Jira remains at the top of the totem pole for what it does.
As an engineer & former manager, there are definitely things I didn't like about Jira like slowness, but everything else I investigated/used were worse in multiple significant ways.
My current company tried Monday. The engineers begged for Jira after that.
There's a worse world - one where you have on prem Jira, Monday, Paper, O365, Confluence, Github, and internal api documentation all out of alignment with the other. I'm begging to just end my misery.
The other thing about jira is it’s configurability. I look for replacements every so often and all will require us to change our process. We have adapted jira to our exact process like I’m sure many teams have.
The thing about products that occupy the kind of niche that Jira does is, the people using the product are not the people making the purchasing decision. They rarely even talk to each other.
If you come to a venue like Hacker News, you'll mostly be getting opinions of the people who actually use the product. These opinions do not reflect the interests and priorities of the people who decide which product to buy.
I wish I knew the answer as well. I believe many managers trained in the art of building software without knowing how to build software are too married to doing processes in a very specific way that's very tightly coupled to Jira, and feel safe and at home with the added complexity it provides, so they vouch for it. But that's just a theory from personal experience.
Jira is as complex as you make it and you can't solve people issues with technology. So another solution won't solve your manager problem. That said, the UI is an abomination that will one day summon the elder gods to reap us all.
Yeah, I've never seen the Jira code, but I've dealt with the API. It is VERY clear the software is a chaotic mess based on the API.
I mean, if the problem you have is that Jira lets you set up overly complex workflows, then a more limited solution that forces a simpler way of working could have a positive effect? I do agree that you don’t HAVE to make things more complicated than necessary in Jira, but obviously some do that anyway.
If the manager wants a complex workflow then they will make one outside the project management software if needed. Except now you'll need to manually track everything as a developer.
Taking away configuration options from most Jira project managers I’ve come across would be a very helpful first step.
Then they'll just have the same requirements outside the software and it'll be on the developers to manually track it. Then communicate it in hour long standup meetings or something. Or bother engineers every few hours to communicate them directly. Complex corporate processes predate Jira and exist in places where the only tracking is Excel documents.
I used Clubhouse at a past job and it was great. I can understand why companies may not be able to move off of Jira, but I can't understand why one would start using it in today's world.
I think it already has been disrupted but no company is going to switch task management software without a really good reason. But I can't imagine and fresh companies are choosing Jira over Clubhouse, Asana, Trello or what I hope to be my company at some point! https://tahsk.com
My new companies aren't. Gitlab is the new hotness
Yeah I've heard a lot of people using Gitlab as you don't need to mess around with any integration and it's all in one place which is really nice. I find actually managing the issues super lacking and frustrating personally though.
fwiw Trello is owned by Atlassian.
The thing about Jira is that it provides much more than just tickets. It's really not even bad at what it does as far as user workflows go, it's just really easy to misconfigure due to lackluster administrative tooling.
I work with a Jira instance that has something like 20 years of history and over half a million tickets. Migrating just the tickets and their comments might be possible, but migrating all the other metadata and every service and automation we've integrated into the workflows (some of which we depend on to be able to work at all) would take months of work if a suitable alternative even exists in the first place.
If it were just tickets and some CI integrations, migrating away from Jira would be trivial, but that's not where all the value is.
There aren’t that many if your requirements include on-prem and being flexible enough to not be only for software development using agile.
A far stretch to conclude that this event can equate to awful engineering.
The rest of this your comment reads like you continue to be naive to Atlassian’s success. I have to think many people do find unique value in their products (myself included), some people don’t laugh rudely when they hear what folks are working on, and I think that shows in the overall achievements of the Atlassian team and product.
I’ve witnessed first hand truly fantastic organizational changes after adopting Jira, Confluence, etc., and I wouldn’t continue to write them off so easily.
And I've witnessed first hand the truly garbage nothing changes after adopting Jira and Confluence - a wasteland of process management through bad automation and forgotten wiki articles with Write Once Read Never behavior.
Nothing Atlassian does is that much better than past tooling, it all comes down to how you want to run your org, what discipline you apply, and where you apply it.
I'm no fan of Jira or Confluence but you'll get forgotten wiki articles, for example, no matter what tech you choose.
Confluence? Forgotten
Git repos? Forgotten
Notion? New hotness, still forgotten
Google Docs / Office 365 ? Forgotten
This is just a difficult problem to solve, especially for organisations growing at speed.
> Git repos?
One of these things is not like the other, and anybody who uses a real editor and VCS but still has to deal with confluence or the rest of the above knows it.
Child poster below going on about markdown etc is absolutely wrong.
You're aware that you can dynamically generate wiki pages from git repos using (you guessed it) confluence, right? The problem is people, not tooling.
If you dynamically generate confluence pages from a git repo, what do you need confluence for? That’s a lot of money you can save.
How? Does it involve a 3rd party plugin? If so that’s DOA for any org concerned about security.
How do you do that without yet another third party addon not even supported by Atlassian? (if you say via the api...)
Agree, that's why I post that the discipline and management is required, not some specific tool - no tool solves the people problems adequately.
Don’t forget Markdown, ReStructured text, asciidoc; forgotten useless garbage.
Agreed, it’s a total waste of time, I’d say our managements love for JIRA is the things that’s really holding us back from really being productive and enjoying our work. It’s such a horrible horrible time suck and offers nothing of real value.
>overhearing someone ask a developer, “where do you work?” When the developer responded with “HipChat,” the other person immediately chuckled and said, “oh — Atlassian... I’m sorry” — and then everyone around them also started laughing.
Wow, This is incredibly mean.
> It’s amazing that this company continues to fall up.
There are still not any knowledge base tools that can keep up with Confluence. For Jira the competition is slowly catching up but there are still a large gap for big organizations. That's why they are still here, their product is still superior to the competition.
Atlassian get a lot of criticism, that's not always justified
I suspect that a lot of Atlassian's criticism is a reflection of their dominant market position. Outside of smartphones and video game consoles, people generally don't spend much time complaining about products they don't use.
For my part, I've spent enough time using both Atlassian products and competitors to find something to hate in all of them. Familiarity breeds contempt.
A former Rally engineer once told me, "Rally did a better job than Atlassian at making engineers think positive thoughts about Jira."
That said, I can't think of a single feature Atlassian released in the past several years that made the experience of using Jira or Confluence substantially better. (I could at least say markdown support if that was ever ported to Jira Data Center.)
The argument for software subscriptions is that it funds continuous improvement, but most of the top complaints from 2011 are just as relevant in 2021.
There's the famous quote by Bjarne Stroustrup, "There are only two kinds of languages: the ones people complain about and the ones nobody uses". I think that holds true for a lot of things.
> That's why they are still here, their product is still superior to the competition.
No. It's because their products are sticky in nature. The tools are used to hold the current state and historic knowledge of the organisation, and even the thought of replacing one of them gives IT manager types the shakes.
> Atlassian get a lot of criticism
Yeah, I think that perception used to be pretty hardcore years ago.
I eventually realised that so many, many companies use this software as a backbone to their company and operations. And for the majority of those, the companies like it. So much so that instead of migrating to a competitor, they move to the new cloud offering.
The search function is atrocious. Some of the most visited, highly important pages will not be found with exact title matches.
You need to start using the asterisk in your title searches...
Is there a way to quickly mark a block as code? Because whatever nice feature it has are completely rendered irrelevant by this lack.
In Confluence you use the {code} macro.
Confluence's code blocks are hot garbage:
* Selecting a language on one code block changes the languages for other code blocks on the same page, sometimes. (I've not figured out the exact conditions on this one yet.)
* Whitespace is not preserved / rendered the same as the editor; we have several Confluence pages with YAML where the rendered version won't parse, but it looks fine in the editor.
Give me Markdown in git any day.
Dunno about all that but it doesn't support inline code snippets. You've gotta use formatted text, which doesn't look good. Seems like an intern project at best.
Huh? What do you mean, isn't
Ordinary text blah blah {{interspersed code snippet}} some more ordinary blah blah text
working, or haven't you tried it?
Just haven't tried it, didn't mean to imply they were false.
Ya gotta use the mark-up text tab to write, and only switch over to the other one ("visual" something?) to check that it looks OK.
Always annoys the heck outta me when I forget to switch back before submitting my comment. It remembers, and gives me the same tab next time I start to comment on any ticket.
Or use markdown backticks e.g. ```
If you are serious about the tunnel between the houses can you provide any info or a link for my bubble folder? I’d love to read about that. Googling was not fruitful.
Same, closest I can find is a tunnel to the harbourfront[0]
[0] https://www.realestate.com.au/news/the-list-australias-top-t...
>It’s only a matter of time before this product suite is disrupted, and it might represent one of the most obvious low-hanging opportunities in our entire industry.
That might be the most disconnected-from-reality statement in this entire discussion.
Whatever you think about the quality of Atlassian's products, they are ridiculously entrenched and about as easy to "disrupt" as Microsoft Windows.
On the other hand, cloud-hosted services can have a CVE patched for all users within a matter of hours (or less). Consider the alternative of frantically trying to get in contact with thousands (or tens of thousands) of companies running your on-prem version and urging each one to install your patch.
Haha, atlassian will not go away. Since it’s not being sold to most people actually using it.
They just sell their feature list to CEO’s and Product Manager/Scrum Lords, and suddenly Atlassian is an absolute requirement.
> everyone is now aware of the awful engineering practices that underpin their products.
This was already obvious to anyone actually using their products.
> It’s amazing that this company continues to fall up, and that the founders have taken on roles as the ruling digital gurus of Australia
It's probably because they primarily target non technical folks. Our IT department has inherited numerous Atlassian products adopted by business units and it takes at least a year or two to unwind them if ever.
In the meantime the just keep cashing those checks.
There's been something amiss since going public, even handy features like dark-mode have been completely shelved... it's laughable really: https://jira.atlassian.com/browse/JRACLOUD-63150
On prem is gone and with this so is my faith in their slow cloud solution.
> awful engineering practices that underpin
And what are these practices?
> assume that there are problems of a similar nature in their cloud service
?
> then everyone around them also started laughing
You know, I'm sure that highly paid dev felt just fine.
Twitter link to a case of the vulnerability being exploited: https://twitter.com/th3_protoCOL/status/1433414685299142660
NIST Link to issue: https://nvd.nist.gov/vuln/detail/CVE-2021-26084
Tweet from USCYBERCOM urging users to patch: https://twitter.com/CNMF_CyberAlert/status/14337876717851852...
Tweet from BadPackets showing where the bad actors are originating from: https://twitter.com/bad_packets/status/1433157632370511873
Nit: I wouldn't say "originating". That's where this specific exploit is coming from "most recently". But it would seem to not be script kiddies and they're listing like 8 countries. I would assume the bad actors could be anywhere, proxying traffic through any number of other places.
Helpful links, looks like failure to sanitize input. Classic.
But on the “attacks coming from”, I’ve never understood putting stock in these. Aren’t these all going to be proxies and botnets?
Failure to sanitize input is one thing, but the bigger issue to me is that, with so many of these Java server installations, that a simple injection can immediately lead to "game over" from a server takeover perspective.
For the bug in question, I bet the vast majority of webservers never need the ability to call unrestricted Runtime.exec(), yet access to that is just one unsanitized input away from complete control over your server.
OS vendors have made leaps and bounds in the past decade making it much harder for code vulnerabilities to lead to system takeover. I'd argue it's time for server code and language runtimes to make it easier to write secure code.
That’s fair. But there needs to be a point somewhere that you just get work done.
I absolutely agree that runtimes, frameworks, and server code should do a better job at trust and sanitization, but you will always get to a point where if you want to get something done, you need to do the work.
I guess I’m skeptical that eval() or runtime.exe could or even should take in lists and configs of what the code is allowed to do and monitor for it during execution. It seems like doing that would add countless issues and complexity, but more so just kick the can down the code to another layer with the same eventual issue.
Atlassian was so kind to update their mailing lists somewhere over the last year or so. Previously, they would email the 'technical contact' of the license about any vulnerabilities. They quietly switched to some other notification system and never informed us about it. Hence we missed the update and got a free Bitcoin miner. Thanks Atlassian, I'll make sure to get your products out of the door as soon as possible.
[edit] Oh it's even better. Their site says 'Note: if you are a tech administrator, you will always receive these notifications.' but they never mailed us. Great job, Atlassian, great job.
Another issue is that they sent out the initial communication on August 25th (which I did receive), but the original wording indicated that it only affected servers that allowed user self-registration. We didn’t have that enabled, so I held off for a bit because the risk seemed lower and our upgrade process is a bit arduous (we have quite a few customizations on the server and need to perform all upgrades on a test instance and validate first) and our instance requires authentication through a load balancer before it’s even accessible.
Then, Atlassian updated the ticket a day later to state the issue affected all servers on the affected versions regardless of user authentication or registration but didn’t send out a follow up communication when they did so. Instead they waited until Friday afternoon before a US holiday weekend to send out another update. So if you weren’t watching the source ticket directly and thought you could wait due to the setting distinction you wouldn’t have known for over a week and you were left vulnerable.
Atlassian should have sent out another communication to all customers as soon as they knew the scope was broader than they had initially thought.
100%, I did the same thing on my side. If shit really hit the fan I could've lost my job because of this as it was my call to not patch. When I went back to the link provided in the email the self-registration part was removed so I looked like a complete tool over zoom when trying to explain this situation to my boss
If it helps you at all, we aren’t the only ones who were blindsided by the severity-level update and lack of further communication. There are several comments on the source ticket calling out the poor communication, and the earlier comments are all asking for clarification about the user registration requirement: https://jira.atlassian.com/browse/CONFSERVER-67940
Both I and another colleague looked at the issue when it first came out and decided we were “safe” for a bit based on the initial communication. Many IT/IS teams were probably scrambling over the long weekend to patch this issue.
Well, I got it. Maybe you specifically didn't get it, or maybe there is something filtering it.
I only got the 'update' from last Saturday, by then it was too late already. Their original advisory was from the 25th, they should have mailed me back then.
If you got the one from Sep 4, you definitely should have gotten the one from Aug 25.
This is unrelated to any mailing-list change, since both were sent from/to the 10991049.xt.local mailing list.
Search for the header entry `List-ID: <10991049.xt.local>` in your Sep 4 email. If it came from that list, the one from Aug 25 will very likely have been lost during transit.
I use their products in the 10-user license program since 2016 and got automatically subscribed to their mailing list back then, and never made some change to the subscription. I'm receiving emails from that list since 2020-10-17.
On the 2020-07-10 I got a mail from them telling me:
```
Subject: Please double-check your contact details for Atlassian
Making sure you don't miss important emails
Hi,
When you purchased your Atlassian products, we asked for the contact information for two types of people in your organization:
Billing contact - A person we contact with invoice and billing information
Technical contact - A person we contact about product changes, security advisories, etc.
We don't want your company to miss out on important information from Atlassian, so please take a minute to make sure your contact information is current. Here's what we have in our system:
```
I'm listed as the technical contact and have been for 5+ years and also get the regular mails to 'verify contact details'. I did not get the Aug 25 email. I did get the Sep 4 mail from that list ID.
Once I noticed I did not get an email, on Sep 3, I checked some checkboxes at https://my.atlassian.com/email But that page also says tech contacts should always receive an email regardless of settings. I have received other security announcements in the past.
Office 365 can't find any emails from Atlassian on Aug 25 when searching using the Message trace tool (which also includes any spam mails, deleted mails, et cetera), so I would suggest Atlassian fix their mailing list.
How big is your organisation? I know it shouldn’t matter but your CS person would likely have reached out if they’re anything like Amazon, Microsoft, Salesforce, etc.
I’ve always found government, sensitive customers (banks, payment processors, healthcare) and big spenders get prioritised with phone call notifications.
However with a deprecated product, the financial impact is so minuscule - leadership won’t prioritise this one unless you’re big fish.
It's tiny, I just want them to send me an email if there is a critical vulnerability. Not too much to ask, I think.
Based on the other anecdotes here it seems most likely that they did and you just didn't receive it.
your CS person would likely have reached out if they’re anything like Amazon, Microsoft, Salesforce, etc.
The only companies that are like those companies are those companies.
In most companies, the CS people don't know what anything in that sort of alert means and will discard it thinking that it's a spam or phishing attempt.
The problem is not that he doesn't work for a megacorp. The problem is that Atlassian screwed up.
I think the claim here is that Atlassian's post-sales account representatives ("customer success"?) would have proactively reached out to the technical contacts of large companies with a personal email - and known exactly what person to talk to, because they stay in touch - because Atlassian is an organization like Amazon, Microsoft, or Salesforce.
I think you're reading it as saying that the helpdesk people ("customer support"?) at a large organization like Amazon, Microsoft, or Salesforce would be trained to recognize a mis-directed email from a vendor and send it to the right place, but I don't think that's the claim being made.
Where did they screw up? How do you know that the mail wasn't lost/filtered after being sent?
If O365 can't find the email and the O365 message tracing does not show anything, it seems likely that the mail was not actually delivered by Atlassian. If O365 looses mails and these mails do not show up in message tracing either (i.e., not classified as spam), we would probably have heard about that by now.
Also, regardless of whether or not I received the mail, the initial mail stated that only authorized users could exploit this. So Atlassian did not inform any of their users fully until Sep 4, whereas they were well aware on Aug 26 that the vulnerability was exploitable by anyone.
> If O365 looses mails and these mails do not show up in message tracing either (i.e., not classified as spam), we would probably have heard about that by now.
Internet email has never been considered a highly-reliable messaging system; its quite possible an infrequent data loss in a mail server would get misattributed to a failure outside.
Heck, even ignoring the unreliability of email generally, in fact, your assumption that it must not occur because you haven't previously heard about it demonstrates how that might happen.
> Internet email has never been considered a highly-reliable messaging system
While that may be true, it seems vastly less likely to be the cause for the GP not receiving precisely this mail... Given that several other commenters only on this page mention not being able to find any evidence of having received this particular missive, William of Ockham would fall over with laughter at the idea that they all just happened to have email system glitches at the exact same mail.
In terms of human communication, I consider anything which is only unidirectional to be unreliable.
It's ok for ad emails. But anything that might cost millions if lost should require some kind of human TCP handshake. Whether by email or phone.
I would beg to differ about email reliability, also see https://datatracker.ietf.org/doc/html/rfc5321#section-6.1 but do agree that everything could get lost for some reason.
But that is not the main point. Even if the email was lost somewhere in Office 365, people were already pointing out to Atlassian that they should really send a follow up on Aug 27:
https://jira.atlassian.com/browse/CONFSERVER-67940?focusedCo...
The follow-up-request was to notify users that the advisory has been updated, not to ensure that customers received the Aug 25 email which linked to that advisory.
Either is was Atlassian's mailing list software which did not attempt to re-send the email to you after it noticed that the connection got dropped (should that have happened), or is was Microsoft which dropped the email after receiving it, but before storing it into the database and assigning it to your account.
You cannot know who is responsible for this delivery error unless you ask them directly.
They absolutely do silent drops of email they consider suspect. Anybody who works with email can tell you this. What this metric is nobody knows outside their walls. Google and other big providers do this too, some regard Microsoft a bit more skittish perhaps.
Partly related https://news.ycombinator.com/item?id=25590846
Heh. We got a monero miner. If we weren't in the middle of an upgrade we never would have noticed. I googled confluence security and saw the CVE.
Did you check your spam folder? Just saying emails can slip through the cracks.
Yes, I did. I also ran an Exchange Message Trace (this is just standard Office 365, nothing fancy) and there was only 1 message from Atlassian in the last 15 days which was the update email that was too late to prevent exploitation. So they did not email me in time.
> The vulnerability only affects on-premise servers, not those hosted in the cloud.
This is a dangerous statement to make and should be revised to say:
> The vulnerability only affects standalone versions of the software, not the managed service of confluence provided directly by Atlassian.
The problem with the former is that lesser technical people, especially directors, might assume they're fine because their standalone instances are hosted on GCP/AWS/Azure, which counts to them as "cloud".
99% agreed.
Reserving 1% because I'd strike "lesser technical" from your final sentence. The misleading quote is simply not correct. It is misleading because it's not true. It says Confluence hosted in the cloud is not vulnerable. False statement that can mislead anyone regardless of how technical they are.
its' misleading and it gives off the notion that the cloud is more secure so you should migrate your instance to our managed "Cloud" version.
But in this case it's literally the cloud product that is more secure.
Let's say 99.5%, because Atlassian hosted offering is called "Atlassian Cloud"
Good compromise.
> regardless of how technical they are
They said "lesser technical people", not "less-technical people". A more technical person might not be able to read between the lines, but a better technical person should.
Why do people say "on-premise" instead of "on-premises"?
Here follows the definitions I am familiar with:
"premise" - a house or building, together with its land and outbuildings, occupied by a business or considered in an official context.
"premise" - a previous statement or proposition from which another is inferred or follows as a conclusion.
(I have the privilege of worrying about this because my company uses Confluence Cloud. It's vastly inferior to our old aelf-hosted mediawiki, but at least it's not an open barn door.)
Just say on-prem. Problem solved!
Really though, "self-hosted" would make even more sense, as companies often deploy such applications in one or more "off prem" environments anyway. I'd hardly consider my company's multi-region/multi-AZ AWS VPC to be my "premise".
(That first one should be "premises", but I didn't proof-read or notice in time to make an edit. Also the last sentence should have "self-hosted". Bad QA.)
It's really tied to specific versions. The fully managed version is always latest.
We got hit by this and had to shut down and upgrade. Atlassian are taking a while to send new license keys.
I hope they can find what they are looking for, because, with the built-in search, I sure can’t.
It is awful, the worst "search engine" which exists. I absolutely hate it and this is the only thing which wants to make me move away from Confluence. When you need it the most, and this happens often, you know that you definitely cannot rely on it. Any data you put in there is lost, unless you have a good hierarchy and know what to find where without relying on the search.
The search engine will happily search any attached pdfs and return those. It just won't search the actual Confluence pages, which seems like it would be easier.
I use this browser extension which seems OK https://chrome.google.com/webstore/detail/confluence-quick-s...
We have an internal search engine. It made Confluence usable.
A colleague who runs security at an ASX 200 company found crypto mining running within a day of the vulnerability being announced. They've since patched and cleaned up the hosts they run Data Centre on. Patch quickly, and check for the IoCs listed in Daniaal's tweet below.
Admittedly low-value comment: Can we appreciate the amazing vulnerability name? Confluenza.
Any historical cases of poor sport lawsuits in these situations?
I consider humour high value content even if puns are the lowest form of comedy
> An OGNL injection vulnerability exists that would allow an authenticated user, and in some instances unauthenticated user, to execute arbitrary code on a Confluence Server or Data Center instance.
For god sake, can we all agree to stop using OGNL at this point? At my previous job I kept having to fix OGNL vulnerabilities on our stack, it was awful.
Don't remember Apple developer portal hack? OGNL
What about Equifax? OGNL
This thing is so freakingly insecure it's crazy.
My employer was bit by this on Wednesday. Thankfully we had Crowdstrike on it which blocked any real damage. But it definitely moved our cloud migration from “later this year” to “later this month”.
Also, not having confluence for a day exposed just how reliant we were on it for day-to-day activities.
> Thankfully we had Crowdstrike on it which blocked any real damage
For someone not familiar with their products, what did they do for you specifically?
For us specifically they blocked the server from downloading more assumedly dangerous tools. Blocked more privilege escalation and blocked crypto mining software from running.
Our teams were also able to do a “network isolation” and essentially bring the server offline quickly, without touching more pieces and possibly exposing our credentials or tokens.
We also had the paid Overwatch protection which is Crowdstrikes 24/7 security monitoring solution which resulted in an actual person emailing half our team at 1am letting us know this was happening and their recommended remediation steps.
I would explain it as next gen antivirus. Looks closer at hashes and heuristics of all data in and out to a server. It seems crazy that everything that is read or written is hashed and compared to a db, but it works.
FWIW, we dumped crowdstrike for Cisco AMP and have been happy.
Security is planning to implement here CrowdStrike in the near future... does it run on every single server?
It runs on every server that you install it on (currently happening at my company - lots of servers owned by lots of different departments need to have it instaked).
Yes.
Got hit too. We are moving to cloud in 3 days!
Tip: adding noexec to /tmp helped.
I am not in the least bit shocked.
Atlassian products are some of the worst glued-together garbage in the industry. The entire product surface area is probably rife with exploits.
Using Confluence or Jira will show you just how much Atlassian cares about its own products.
I'd love for this to be the straw that breaks the camel's back and makes IT/infosec orgs move away from this bilge.
I have no idea why you're being downvoted - this is true.
Atlassian produce some of the worst tech on the planet. Trying to administer this crap is horrible.
And don't get me started on how many project managers spend all day staring at Jira tickets instead of actually talking to their teams. Management-by-Jira is a disease, a symptom of bad organisational culture.
I'd rather get a fully-formed Jira ticket, than a terse three-word titled empty ticket, that the PM will explain in a 9AM Zoom meeting. Just me though.
But jira is only a tool right? Blaming Atlassian for a poorly led organization seems slightly misguided.
At some project size, measured either by software complexity/interoperability or user base, you will need a tool to manage issues and tasks.
What you're talking about is an organization where developers are not empowered - but even empowered developers need an issue tracker or a board of some description.
A "management by jira" culture will not be remediated by tooling.
A good tool cannot guarantee good results, but a bad tool can shape its user's behavior in bad ways. The same individual without that tool might have behaved differently. I have seen this in what I call design-by-ticket where all the why and how for a design decision, including design by committee meeting notes, get put in jira tickets.
Totally agree with your first two sentences.
However, the final part has nothing to do with jira IMO. You would have the same behavior in these organizations regardless of tool.
The dysfunction you describe goes way beyond a single or set of tools.
It is hard to say for sure, but this organization used configuration controlled documents for design documentation before atlassian came along. When they made the switch (and hired oodles of graduates) suddenly about half of them ignore confluence or latex for design, because "jira has markdown". Oh hurray, we have a bad typesetting language mixed in with our bug tracker, lets shove our entire design in there! It is possible this is the influence of the graduates we hired, but it felt like everyone got there together while playing with the new toy.
Can't but help to feel that what you describe is a breakdown of both organization and work process.
It wasn't atlassian that "came along" - someone penned down an agreement and, from what it sounds, there was a lack of clarity both in regards to current and future principles of work and collaboration.
What we can do, as devs, to protect ourselves from the madness you describe is to be explicit about our work processes and their purpose. Nothing much, but just a set of agreed upon principles and ways of working which should have nothing to do with tools.
What you end up with, by being explicit, is for having "something" to be supported by one or several tools.
Makes it easier to evaluate, select and change tooling that is fit for purpose.
The thing I've found is that Jira has so many fields on its tickets that beg to be filled in, that PMs start filling them in, and before you know it they're all mandatory and must be filled in. And organising that much state in the tickets becomes a full-time job, and so the PM ends up doing that - managing the state of Jira rather than managing the state of the project. The two become synonymous when they're not. When they inevitably diverge all the usual problems of project management get exacerbated horribly (in part because what should happen is all the state in Jira gets thrown away as it doesn't reflect reality, but that's a huge sunk cost so no-one does it).
So yeah, I have found that the tool shapes the process, not the other way around.
It would probably work the other way around if every organisation built their own project management tooling around their own processes. But that would be insane.
So basically you are describing an organization of people that don't really understand what they should be doing.
People led by a tool and not the other way around - and you blame the tool?
I hear what you are saying, and I've seen the very symptoms you're describing - I've just stopped chalking it down to the tools.
It's a symptom of something entirely different and much more challenging to deal with than a change in tooling.
If you provide a tool that let's managers easily and arbitrarily increase the requirements on their employees, over time they'll continue to do so, because it's a management tool and their job is to manage.
I experienced this phenomenon when I was a designer / CNC programmer. We had a form for requesting a part to be designed and machined. It had a box for tolerance allowance, where the person requesting a part could specify how tight all the tolerances should be, and we had recently added in 0.0005 inches to the options, at the request of a customer. I left for a month to do training at another location and came back to find a ridiculously long backlog of work. The manager who'd stepped in for me had decided that tighter tolerances would make better products, so was selecting 0.0005" tolerance for every feature on every part.
To make up for how ridiculously slow that made the machining process, he tried to micro-optimize work flows, readjust hours, push machine operators to "work harder". He wasn't a bad person, was an okay manager, we'd just given him the option of doing something stupid, and he'd done it then tried to use his managing skills to make it work.
If you give someone the tool to do their job, make sure that misusing it feels hard.
He obviously did not understand what his job as a manager is about.
Why on earth would a manager be allowed to set tolerances like how you describe, tool or no tool?
It makes no sense and is first and foremost a management and cultural issue. Second a work process issue. Solid third, one of competency. Probably ways below numerous other problems lies the tool.
You obviously needed to be able to set tight tolerances for some work, so… you seem to need the setting on occasion.
These stories just blow wind in my sails!
If you have management like this (american?) you must be measured like crazy - set goals based on metrics that will push things in a direction you see effective.
> Why on earth would a manager be allowed to set tolerances like how you describe, tool or no tool?
Because the tool allowed it. What you’re failing to grasp is that UI has a massive influence on how humans behave.
People see empty fields and think they should fill them out.
Jira very frequently encourages over-specifying things by the ticket filer (it doesn’t hide fields by default and “helpful” human behavior is to provide as much info as possible).
The second order effect is that the project managers realize they can add fields and people start filling them with data. This is great for visibility without having to poll all of the employees! Except now it’s garbage data because the wrong people are filling in the data or they are bad guesses and inevitably that rolls up into a report that causes problems later.
Jira is a bad tool because it misleads both people and product managers into thinking they can get data from it that they realistically can’t.
> He obviously did not understand what his job as a manager is about.
> Why on earth would a manager be allowed to set tolerances like how you describe, tool or no tool?
He's not the expert in the field, I am. Normally, I would have vetted the work orders and fixed it before hand. This is similar to managers in the software world, where the team lead or senior engineer would say," No, we won't do that, it's a bad idea". He never should have been exposed to an option that could screw everything up so badly, but I mistakenly left it on the form. He was just trying to fix what he perceived as a potential problem. He was used to making small changes to work orders to save money or get a more refined product.
The biggest problem with our company's structure is that the operators, for whatever BS org reason, don't have the "pay grade" to tell him to pound sand. Managers need to sit below engineers, in my opinion.
> you must be measured like crazy - set goals based on metrics that will push things in a direction you see effective.
In my sector of manufacturing, margins are king (regardless of locale). If you can cut 10 seconds from an operation, or find a tool that lasts 20% longer, you can save tens of thousands of dollars a year, so we track everything.
Again: describe a set of work principles explicitly that all agree on - it's a human communication and collaboration issue you were faced with.
A tool is not responsible for this breakdown. Lack of clarity around ownership and responsibilities is, by the look of it.
IM humble O.
I'm willing to entertain that the teams using Jira are just making worse choices about the work principles they agree on than our teams using gitlab issues, and that it is coincidental and unrelated to the tool. Surely you agree though, that in general a tool can adversely affect user behavior? The agile manifesto said to value tools and process es less than individuals and interactions, not to discount their impact altogether. Developing software in a single paradigm language when your domain is better suited to a different paradigm is a horrific blunder I have witnessed twice in my career. Developing giant systems using vba macros in excel is a common mistake you hear about. I can however believe that the teams that selected jira over gitlab might have done so because they already bad this use pattern in mind, and not because the tool encouraged it.
I’ve worked in fantastic settings, truly following the agile principles, using jira as a tracker and board.
I’ve been places with many different tools and people.
Guess what, it’s ALWAYS about the people.
If a team is following principles derived from or resembling the manifesto, the tool will be shaped accordingly.
Jira, as an example, sure can be shaped, and I’ve done it myself several times - remove cruft, keep it simple, change if needed.
many organizations are overwhelmed with inexperienced new graduates; they end up doing what they want or how they did things in college projects.
then they get promoted, and may never learn / experience why a task / defect tracker is not where you store requirements / design.
(note: the above comments are for long lived software only. it you rewrite your web site from first principles every you, do whatever. it doesn't matter)
Bitbucket recently has shockingly poor reliability. Quite often you see nothing on the status page but see other people having issues on twitter. We've nearly migrated everything to github, plus github has better features and more powerful.
They've been doing a large transformation recently to put it on a new platform. Not saying that's an excuse, but it is an explanation for some of the problems they've had.
Yup, moved from onprem to cloud afaik. Turns out cloud hardware has trash reliability and slow (particularly disk) Big shocker here
Even after this we had issues. Pull requests can take quite a while, diffs not working, git slow or timing out.
I once said this too.
Then I tried a bunch of their competitors. Still stuck with some of them.
Sadly, some of Atlassian's products - namely Confluence and Jira - are the best in the business.
Those complaining below about PMs staring at JIRA all day... well, this is a problem with PMs, not JIRA, and it happens even if they are using other work management tools. We created a middleman position in our business to deal with the stuff we didn't want to - tracking work, getting requirements, etc - and we must reap what we've sown. They become obsessed with the management stuff because that's why they exist, and they will fill their time to justify their existence.
Any suggestions on what to use instead of Confluence? Need to run on-prem, it's mostly the wiki-like features I'm interested in.
> Need to run on-prem, it's mostly the wiki-like features I'm interested in.
Since you are looking mostly for the wiki part there is Dokuwiki which is magnitudes better at being a wiki. Remember, wiki is derived from the Hawaiian word for quick or something to that effect and whatever Confluence is it isn't quick.
Don't know how well it will hold up under scrutiny if black hats gets a reason to swarm over it, but unlike Confluence you can hire someone to patch the guts of it if necessary.
Edit: I have no reason to believe it is worse than anything else, I'm just pointing out it probably hasn't had so much exposure to help it harden.
For a non-technical user group you likely want something more WYSIWYG than Dokuwiki.
Maybe. But I have a hunch that we are severely underestimating huge parts of the workforce.
I mean: ux discussions often feels like they assume users are a separate species somewhere between ordinary humans and chimps when it comes to intelligence.
I have some experience with training users and I have only given up once.
In my experience, it depends on the industry and company how competent their users will be. I've been a training (back when thinngs were in person) where a user started getting irate because their login wouldn't work to our software. The trainer walked over to see what was going on, and the user was trying to put their login credentials into their bank website instead of ours.
These were sales people.
So often, somewhere right above a chimp is where some of your users are.
I fully agree that you can train users for a lot. But the question is if it's worth doing so in the specific case. And wikis often already have trouble with people not using them enough, making them seemingly hard to use doesn't help.
Don't forget that in many ways some real wikis like Dokuwiki are simpler and more user-friendly than Confluence:
- you can edit a paragraph without locking or creating conflicts for the whole page
- it is much faster, and as far as I can see the observe–orient–decide–act loop is a real thing and should be taken into account
- much better wiki syntax (compared to old Confluence)
- trivially extensible so you can create forms and other helpers to make it simpler for users to do the right thing
- if one absolutely need it I think there exist at least one wysiwyg extension for Dokuwiki
I am considering going with Bookstack Wiki for my company. But I am not a Fortune500 company.
I would say it is very darn complete, perhaps without the syntactic linking that Confluence has. The only thing missing from it, is a very solid backup and restore method from the admin panel. The authors want users to rely on database backups and file level backups, that must be handled manually. Essentially saying "not my problem".
> The authors want users to rely on database backups and file level backups, that must be handled manually. Essentially saying "not my problem".
Just a note on this, this is something I'd like to build in eventually. It's just that each layer of our own we add upon the underlying methods increase the risk of error in something fairly critical.
A simple command-line based backup solution wouldn't be too difficult to add, but restore and admin-panel usage are more significant challenges once you get into it.
Mediawiki has worked well. I’d be curious to hear others experiences with it.
VisualEditor is an extremely good WYSIWYG interface. The wiki is able to scale well (project sites are unlikely to approach Wikipedia scale). The API is useful. Wikitext editing gives power users a lot of flexibility, though it’s not as popular as Markdown.
Access control & edit-publish workflow options may be too limited for the desires of some project teams.
My company is currently migrating off of Confluence due to the ending of the on-premise license. We found a combination of a simple web-based knowledgebase app for public content and Office 365/Teams -- which we already license.
You could put Cloudflare Access (with Tunnel) in front to shield against exposing the instance directly, I’ve used it in the past for other products that has been good.
I migrated every confluence instance over to XWiki, since the Australian backdoor law[1].
http://www.xwiki.org/xwiki/bin/view/Main/WebHome
https://xwiki.com/en/try-xwiki/
[1] https://www.wired.com/story/australia-encryption-law-global-...
You can try Wiki.js or bookstack, both are open source and nice
Thanks for suggesting BookStack [1]. I did initially create the project as a free open-source alternative to confluence as I didn't want financial approval being a limiter to documentation collaboration.
One of the main things to consider with BookStack is the semi-fixed Shelf > Book > Chapter > Page structure (Where pages have content, chapters & shelves are optional, and Books can be within multiple shelves). For some this structure is limiting, For others it helps drive organization.
What target group? Devs? Po? General org?
I'll just go with "Yes" because we use the same Confluence installation our entire organisation. Spaces might be configured differently, but we can have all our documentation in one system and link across space.
There really isn't that many alternatives, SharePoint maybe, but then you're just suggestion something worse.
Biased but I'm actually building a competitor (V1 is almost ready) to Confluence for medium to big organizations. But I don't understand your requirement for on-prem. That's clearly not an advantage from the security point of view. Apart from Quip, Sharepoint and Confluence (soon stopped) I'm not sure there is any commercial knowledge base tool that are available on-prem. The only thing that you can hope for, is "bring your bucket" in which static resources can be uploaded in your S3/Google/Azure bucket and some managed instance (like Atlassion Cloud) for the app. But on prem is going to be something from the past honestly
Being able to self-host is a hard requirement for many organizations. Especially for tools like a wiki, which may contain proprietary/secret business information.
The last time we reviewed the Atlassian Cloud hosted products, they did not meet our security needs (requirements include isolated tenant from other customers, customer managed keys, etc) though they were much closer than a few years earlier. We also review the general security practices of the company (for example to make sure they implement a secure SDLC and follow other security best-practices).
Plenty of deployments will continue to be on-prem, cloud is not perfect for 100% of use cases.
Example: If you are a semiconductor manufacturer, you probably are not going to be storing all the documentation about your bleeding edge fabs on a cloud provider.
PII, PHI, classified data, controlled unclassified information, proprietary trade secrets, a requirement to host user data in the country the user lives, a company is massive enough to own data centers and may as well use them. There are so many reasons to self-host. Unless your service is as big as AWS/Azure and can afford to build a private cloud in a location of the customer's choosing for sufficiently large customers, "only cloud" isn't good enough for many use cases. You're putting your own convenience as a developer ahead of the needs of certain users. Which is fine, I guess. You don't need to serve everyone. But you shouldn't be out there acting like self-hosting never makes sense. You may as well ask why some facilities have their own generators and their own wells and don't just universally hook into the public grids. Some applications have requirements for hard perimeters and self-reliance.
The issue is that to put company data on your server, we need IT security people to sign off on it. That takes years and a lot of budget, just to get that authorization. It also likely comes with restrictions, like not being able to use it for very sensitive company data or government data.
In short, that’s not happening for a wiki.
>I'm not sure there is any commercial knowledge base tool that are available on-prem.
XWiki?
I personally do not categorize them to equivalent to Confluence, Sharepoint, Notion, Quip, ... but if you do, yeah there are few wiki software which are available on premise.
What can confluence do what XWiki cannot? But i understand that you try to promote your cloud offering ;)
I'm not trying to promote my cloud offering. I would like to be able to offer self hosting, but we still haven't found a way to do it. This is too much hassle for everyone. There is a tradeoff between ease of use of a software and the security process. A collaboration tool which is difficult to update and thus will not evolve quickly is an issue for its adoption and for its benefits.
Well first its interface, if a non technical user can't user the tool that's going to be a big problem. Confluence has not the best interface but still better than Xwiki. The best the interface the better the adoption as a whole in the company. And for knowledge sharing adoption is critical. That's actually why Notion has been such a success recently. Even though their product is average, people want to use it, because they like the interface.
I don't have the time to do a full comparaison of other features, but wiki tools are in usage very different than collaboration tools. Can you have the list of last consulted documents ? Can you embed external documents ? ... Kind of the same things as Slack vs IRC
>I don't have the time to do a full comparison of other features
That should be your first prio before anyone locks one's information into your system...XWiki can import and export confluence and even wikimedia data, i never use a system when in cant import/export to the the next best product.
Confluence can take text and turn it into a black box format shoved inside a database that you can't edit with a real editor, then shove that text into a black box unstandardized version control system shoved inside a database! Take that, other tools!
> black box format shoved inside a database
Are you drunk? Who tf ever wants that?
Sorry, must have lost some context... nobody wants that if they know what they are doing. I don't want it. I'm saying this is the reality behind confluence. (and I don't like it either!)
Atlassian products are garbage.
So why are they so popular? Because Jira is a wet dream for mediocre micro-managers (of all levels), allowing them to manage by ticket, instead of lead by example.
Hit the nail on the head there.
New thing? Let’s open a new JIRA project and prefix with some random shit show workflow customised by someone who was clearly asleep or incompetent!
Yup, have been in that exact situation. It was literally mind-boggling.
Ow.
The linked proof-of-concept [1] demonstrates bypassing the OGNL blacklist by using this to do reflection:
> ""["class"].forName(...)
as opposed to:
> "".getClass().forName(...)
Does anyone know why this works in OGNL? It does not appear to be valid Java syntax.
[1] https://github.com/httpvoid/writeups/blob/main/Confluence-RC...
Edit: Oh apparently, it's just a feature of OGNL: https://commons.apache.org/proper/commons-ognl/language-guid...
Never used it, but a quick perusal of its Wikipedia article mentions that it was a rewrite of something else using ANTLR, which implies a separate syntax.
I look up to Atlassian. Somehow they continue to easily sell even though so many hates it. I don't know what the secret sauce is... but I want it.
This is the power of meeting the "needs" of business side suits who don't know how to use git or a real editor. So many times I've gotten pushback about writing docs in git instead of in confluence because "what about the non-technical people, what if they need to edit something?". So the lesson learned is that if you can use your proprietary vendor lock in to trap a bunch of C-levels via stockholm syndrome you can just keep failing up no matter how shitty the actual tech on your product is.
What's ironic...is instead of educating and offering a workable alternative, you actually made the problem worse and the sale easier for Atlassian!
No, the rest of the company shouldn't be required to enter the complex and esoteric world of Git and fire up a terminal + a bloated code editor and deal with merge conflicts just so they can collaborate on a simple text doc.
This reads like a horror story I'd find on the landing page of some Saas tool under a heading that reads, "The Problem"
> No, the rest of the company shouldn't be required to enter the complex and esoteric world of Git and fire up a terminal + a bloated code editor and deal with merge conflicts just so they can collaborate on a simple text doc.
Instead, they just won't update the doc at all and will pester the kind of person who does know how to use git until they do it for them. Then the doc will sit there dormant and un-updated until the process repeats.
> a bloated code editor
Have you ever used Confluence? SMH.
All those defending Atlassian need to go read this other current front page hn article: https://news.ycombinator.com/item?id=28414308 and probably lots of other articles about good documentation.
To be fair, I understand what you are saying, but the problem is you are trying to meet the needs of suits, while I'm trying to meet the needs of technical teams. I can acknowledge that git can be daunting for suits, and probably not the correct method for them to write docs, (e.g., I'm not saying force the suits to use git, even though, with a web interface like GHE or gitlab etc, this is actually quite easy and visually intuitive, no terminal or (laughing) "bloated code editor" required) but it seems the "suits side" of this argument doesn't want to acknowledge that the needs of technical teams aren't being met when they force their lowest common denominator tool on the entire org.
It's like Microsoft in the 90s, everyone wants to hate on the company but their sales department just laughs and pens another huge contract
Atlassian doesn't have a sales department (or at least this was the case for well over a decade, perhaps it could have changed now).
I get what you are saying, but it is a semantically spin.
They have pretty much everything in the package. You don't really have a lot of alternatives out there that are in the package.
And look at the stock. If someone told me it would ever reach $180, would have been shocked. It’s now $384. And it’s outperforming the expectations all the time.
All the people who claim it is awful software, they ignore how many people love the Atlassian suite.
Not that many people “love” it, that’s why it’s always surprising how well it does. It’s pretty unpleasant to use but there isn’t really anything else out there that’s so well integrated so they keep winning despite the pain.
That's one of the selling point of Saas compared to hosted instance honestly. Some company think that having Confluence hosted internally is going to increase the security. But this is wrong. When you rely on a Saas provider. The provider has people who monitor the infrastructure constantly whereas when you hosted on your own server, the confluence instance is just one of the many services that they manage. And even if some company will be very reactive to events like this. The majority of companies will be much slower.
And in addition to that. When you use Saas. Security is a top priority, a Saas provider can't allow to have data of its customers leaked on the web. Whereas once again when it is internal data people will be less cautious
This isn't always true. Using a SaaS is outsourcing these concerns, and sometimes you're outsourcing them to someone who will do better than you would and sometimes worse. I've worked on a couple of SaaS where security was absolutely not top priority. Especially in Silicon Valley, organizations often value growth over sound processes, fully staffed security teams, and managing tech debt. Many a SaaS has leaked customer data and survived, so many think they CAN allow that risk.
I didn't say that it is always the case. The same argument you use can be used to talk about companies who are going to self host Confluence.
I agree that a lot of Saas startup are going to neglect security. But here we are talking about Knowledge base tools Saas companies. This is not some standard Saas company. They know they are in charge of company internal secrets. Or at lest I hope
Any time a SaaS gets compromised there's a similar comment here about how obviously this is going to happen when you give someone else your data, and it should have just all been within your own firewall, unexposed directly to the Internet.
I mean right this minute there's a privacy-focused SaaS on the front page for not being as private as everyone thinks. There's also a network hardware vendor on the front page for including back doors. A philosophy like "SaaS vendors know they can't allow security breaches" is really glossing over the need for layers of security and knowing that it's ultimately all on the trustworthiness of specifically who is involved.
If you can afford to not expose it to the internet obviously you are going to have better security. But this is not always desirable talking about wiki software.
I can't disagree with you. But you can either deny that the average Saas is more secure than a forgot Confluence internal servers exposed to the internet
Well yes, public wikis are one thing. But before you were talking about protecting internal secrets.
It's the selling point of self hosting. My jira is behind x509 client certs, others I know are behind oidc connections. You need to be an authenticated user to even load the page. There's two layers of protection from two different companies.
If you are running it accessible from the outside maybe.
But a big point of hosting it internally is that you don't have to.
There is no “outside” and “inside” anymore. Have you ever had a penetration test on your network?
The difference then is shifted towards whether it is a targeted attack or not. And that is a huge difference.
That difference would undoubtedly have saved every single company that was affected in the current story.
The hackers will see how bad our team burndown rate is
You just made my day. Thank you.
I spent years "working on" (battling) our own company-hosted Atlassian suite. I'm a software engineer / architect and was thrown admin powers to get a project up and running.
It was constant a battle of "the critical basic feature you need in this micro version is broken" and other critical functions being hidden in random places.
I applied to their engineering team citing my experience and ability to help with a lot of these things, but never even heard a response.
Current alternative software suites I've seen are beyond terrible or generally non-existent / missing major features. I'm sure there's some "pretty SaaS solutions" out there from a startup that charges exorbitant prices, but I don't believe their back end or security are going to be any better.
Why are internally hosted instances even available on the public internet?
Because you might need it to share documentation with customers. Confluence isn't just for external documentation.
Confluence, at it's core, is just a wiki. Sometimes it needs to be available online, sometimes it really doesn't.
If you’re ok sharing things externally why self-host at all?
> If you’re ok sharing things externally why self-host at all?
You're theoretically more in control of the data, which may be a legal requirement in certain jurisdictions and/or industries.
Due to the "TOLA" Australian law, all Atlassian products should be avoided if you care about being in control of the data.
https://www.zdnet.com/article/whats-actually-in-australias-e...
Atlassian is not HIPAA compliant, so many are forced to install their tools on on-prem.
Which is one of the reasons why Atlassians discontinuation of their server offering is so problematic. There's a number of smaller businesses which could use Atlassian Cloud, but between HIPAA, Schrems II and GDPR that's currently not possible, and the datacenter license is simply too expensive.
Exactly. Our customers do need to authenticate to read anything in our Confluence installation. Ideally there's nothing critical, just stuff which is considered private.
Legally many of our clients require that their data, all of it, secret or not, reside within the EU.
Currently cloud is not so hot, due to Schrems II. During the last six months we migrate a number of customers on-prem, and only one is building out their stuff in AWS.
Cost? Availability of oodles of storage? Integration with other on-prem systems (such as Active Directory) which maybe you don't want to directly expose by itself.
There's a fair few reasons other than this, it's not unthinkable to host servers yourself if you already have other servers on-prem anyway.
Because you’re only ok with sharing with your partners, not with a cloud provider.
Because it’s still cheaper than cloud, it’s still not putting your data on someone else’s servers, and it’s still not being beholden to the cloud provider’s planned and unplanned outages.
For those that believe in the zero trust model, don't all apps and services become exposed to the public internet?
They are generally exposed through a proxy that sits in between. If you don't authenticate, you can't send a request to it at all.
(This is opposed to the lazy model, where your aplication is fully exposed to the web and you click log in and it redirects to SSO - if there is a vulnerability that doesn't require authentication you're already compromised)
The proxy will handle sign in and passes traffic to/from the webserver backend, and you should not be able to send a single HTTP request to the underlying application without the proxy capturing authentication and who the user that sent the request was.
Do these proxies encrypt the traffic? Since they would handle authentication, I am guessing encryption is used. We may be getting into semantics, but at that point, is there much of a difference between a proxy and a VPN?
I had the impression that in a zero trust environment all apps are required to be hardened to the point that they are deemed safe to be exposed to the publc internet.
VPNs are a very different technology. The amount of network traffic alone is a huge difference.
You may be confusing inbound and outbound proxies. If an inbound proxy is put in front of a server, it is able to be extremely hardened without exposing the webserver, and only allows traffic in extremely limited ways. This frees up the webserver to be more open to the internal network, obfuscates information about the webserver to outsiders, etc.
You may be thinking of an outbound proxy where all of your web requests travel through it in order to provide web filtering, protected DNS, and other protective services to internal endpoints and clients that are already on your network.
A VPN is a much deeper connection than an inbound proxy. It typically requires the external endpoint to be trusted, which is the complete opposite state of an endpoint connecting through an inbound proxy. It then allows the endpoint to operate from a position of trust within the whole internal network. Patches can be updated, endpoints can communicate with each other, and the full network traffic of the endpoint routes through the network. With a VPN you even wind up going through the outbound proxy typically!
YMMV though, not every org is setup exactly like this. I’m making gross generalizations about things. Definitely do some web searches on the differences.
All apps authenticate every request thus making lateral movement harder. IP addresses are not authentication. OS's don't control IP traffic in any meaningful way: confused deputies all the way down.
So that users can be at home or on a mobile device without requiring them to have VPN.
But so that you still can ensure data-locality or run a customised instance e.t.c. if you have requirements around that. Plus licensing is approx. 40% of the full SaaS cost at scale so may be cheaper to deploy that way.
But why are they not using VPN?
common reasons could be :-
- Cost, VPNs and the hardware to run them can be expensive
- Single point of failure. If you run all your remote access through a VPN gateway then you run the risk of disruption if it goes down. Of course you can implement redundnt/multiple gateways but that increases cost.
- Complexity for B2B setups. If you're exposing an API and you want third party services to access it, it can be more complex if there's a VPN involved.
All that said, I still wouldn't run something like this (or indeed most services) directly on the Internet as it's a single vuln. away from problems, however I've seen plenty of services directly visible on the Internet for these reasons. You can spelunk around one of the search engines like Shodan or Censys to get an idea of how many people run application services directly on the Internet.
A pair of openvpn servers will run you about $100/month in AWS. Sure there is some overhead in running them but not anything more than any other server. Maybe I'm just a jaded Sysop but this stuff is networking 101.
Sure and if you're on AWS you can use their managed offerings. There's a variety of ways of solving it, but it depends on people seeing it as an issue to be solved.
I think, unfortunately, what a lot of people take away from "zero trust networks" as a concept is get rid of all bastions/VPNs and firewalls, but that ultimately leads to the topic of this article...
If all of these are reasons, then you should use the cloud version. In other words, if your project management solution needs to be hosted for some business reason and you can't afford all of the maintenance required to host it properly, then you aren't charging enough for your product.
As well as all the other great reasons listed, you might not want users to have a VPN if for instance they are an external user (e.g. if your client's documentation is on your confluence instance and you want to give them access, it might be a giant pain and bad customer experience to force them to sign into your VPN in order to get access to their documentation).
Some people think VPNs are harmful - see the Google BeyondCorp model.
our VPN is a PITA to connect to.
being connected all the time sucks for zoom meetings as the vpn server is on the other side of the continent.
I love having access to most of my work tools outside the vpn - github, confluence, jira, aws console.
Probably because they’re on a mobile device or essentially monitoring 24/7 would be my guess.
Same reason as why Wikipedia or Wikia or other wikis are public?
Why is Confluence so popular anyway? Why not just use any free wiki software?
It's easier for non-techs to pick up.
Confluence is often where the long-term docs for product/design oriented team members end up living, or at least being linked.
The easy two-way connection between Jira and confluence uses syntax any social media user will be familiar with, so non-techies can link the 'what' with the 'why' in a task before engineers even see them in a grooming session.
Anything that moves documentation and ticket preparation effort away from engineers/tech leads/team leads has a significant hidden saving.
Because most free wiki software is kind of bland and terrible. Don't get me wrong, they are amazing for what they are but they don't scream "professional".
But actually that's not the key point. Nobody buys just Confluence. That would be silly. A bland and terrible (but free) wiki software is definitely better than Confluence.
People buy JIRA. And then you've bought into the Atlassian ecosystem, and you want the nice tight integration with your wiki software
Is there a simple way to test if I've applied the mitigations properly?
There are several exploits on github, e.g. https://github.com/march0s1as/CVE-2021-26084 This one opens a shell but I haven't tried it myself.
Atlassian has been producing remotely exploitable code for a decade now.
https://www.cvedetails.com/product/8170/Atlassian-Jira.html?...
I would also say based on experience that if they tell you that an exploit can't be used against any of their other software that you shouldn't ever believe them.
Use the flaw to deploy the patch, I say.
Seems odd that the Priority is "Low" on the ticket
Would sure be a shame if the only way to fix this was to delete all copies of Atlassian software from every computer worldwide
I know a guy who said “We don’t show up on Shodan because Shodan only groups by IP and does not know the VirtualHost, we’re fine”
FYI: Shodan also does monthly hostname-based scans of the Internet where we set the "Host"/ SNI headers. We use our own DNS DB to grab a list of hostnames/ IPs to launch scans of:
https://www.shodan.io/domain/ycombinator.com
At the moment, I think we're checking around 600 million hostnames.
Is that DNS DB publicly accessible?
Yes, via the API. Btw all of our websites are entirely built on the same public Shodan API that everybody else has access to.
Atlassian software are some of the most annoying to self-administrate. avoid it if you can.
Can anyone comment on what the value of this attack is to the attackers?
At the very basic, almost any attack can be monetized through resources (crypto mining, DDoS-as-a-service, selling access to the machines to other criminals) or extortion (ransomware, threatening to expose data), at scale and without the attackers really having to care too much what they hit.
If they devote more time per target, they can also go after specific data, e.g. for espionage or insider trading.
One compromised server can also serve as a foothold ("oh, you have a service account with all permissions on that server? nice!") which then allows all of the above to be launched against a bigger part of the infrastructure.
Arbitrary code execution in an on-premise server? You can basically stage an attack on any other internal resources (core infrastructure, databases, endpoints) that are visible from there, with the benefit of already being behind at least one layer of firewall/security.
> Arbitrary code execution in an on-premise server?
That doesn’t explain what the benefit of the attack is. It just explains that it’s an effective attack.
How is anybody supposed to be able to answer that question, then? It obviously depends on what other services are running on that server, and what your cracked account has access to.
Access the janitor's account on the facilities Jira, which is all that runs on an old Pentium II in the broom cupboard and you get one set of benefits; access the CFO's on the Big Money Server and you get something else entirely. How on Earth did you manage not to realise this by yourself?
One of the companies I know use it for HR, payroll and account receivables. If you hack into that, you can get a lot of information.
How would that information be useful?
Everything from identify theft, to phishing to industrial espionage.