Tell HN: The German job market is crashing
If we looked at the German Job Market as if it were the stock market, we would say that it's crashing! On the following link you can see my pet project where I have been scrapping the major job offer portal in Germany for over one year. In the last two weeks it has lost 33% percent of all posted job offers and it keeps dropping as a rock :-(
Dashboard: https://jobmarketanalytics.com/#months=%2212%22&technology=%...
Source Code: https://github.com/petracarrion/job-market-analytics
Slide Deck: https://petra.carrion.io/job-market-analytics/slides/
Important! If it is offline, please refer to this image: https://gcdnb.pbrd.co/images/ElMG6yee19KT.png?o=1 For me, the OP's interpretation is very questionable or at least hasty. The first reason that comes to mind is that there is less recruitment at the end of the year. Because in the above statistics (due to lack of data?) only the data within the year is possible, such rhythms are not recorded. I talk to HR people from different companies (in Berlin) from time to time and they still complain about the lack of skilled workers.
Lastly, there is an armada of economic institutes in Germany that do not foresee a "job market crash" either. Jobs are a lagging indicator and the German economy is still growing. Only next year will there be a slight recession. P.S.: For a good statistical analysis, the data source should be mentioned (which website), different sources (e.g. different portals, different source). A data lake otherwise does not make much for an alarmist analysis. Have you checked the chart linked by OP? It has data for the same period of the last year and there’s no such drop so it disproves the seasonality thesis. There could be something else of course. But also possible that some companies are being hit by energy crisis. It may disprove it. Another possibility is COVID affecting the numbers from last year. I'd like to see monthly numbers for at least a couple political cycles, or better yet for several stock market cycles. The data seems to be really in a 365-day window, and on the 1st graph of the dashboard, the beginning and the end seem to be on the same level... > Only next year will there be a slight recession. How do you know that? We talk about prognosis from different entities. https://www.tagesschau.de/wirtschaft/konjunktur/eu-kommissio... EU commission thinks, it will be a loss of 0.6 %. 0 the y-axis otherwise the chart is misleading. Good point, now the y-axis starts at 0 but even now we can appreciate how abrupt the fall is. The fall is abrupt, but not at all surprising. Imho, the data isn’t really sufficient to lead to the conclusion that the job market is crashing. First of all, judging by the overview chart the decline is steep, but it is currently roughly where it was a year ago. Given that we don’t have a huge tech ecosystem in Germany, most people aren’t employed at tech companies, they are at traditional companies. And for them it’s not that uncommon to see less newly opened positions towards the end of the year. Given that you just started scraping a year ago that seasonality hasn’t been captured so far, but I would expect the job portals data to support this. Beyond that, many companies dramatically increased hiring in the later (post-) Covid area. It would not surprise me if some companies over hired given the currently harsh economic environment. Since laying off people isn’t as easy as it is in the US, you stop hiring to make up for it. Also, given my limited experience from the hiring side of things, the demand still seems to outpace the supply of reasonably qualified individuals. According to Statista, Germany had 15k CS graduates in 2021 [1] and turnover of experienced employees is usually relatively low. So considering this and the already declined >100k total openings is still not a real reason to worry (at least based on this data) [1] https://de.statista.com/statistik/daten/studie/789519/umfrag... Since the jobs are scraped and not fetched through an API, I'd also check whether the structure of the site changed in some way. I'm very suspicious of the data, because a 33% decrease in less than a month is unreasonable. Even at the height of Covid, there wasn't that much of a drop-off. My money is on the script being broken. Yeah, our observations at GermanTechJobs.de also don't align with the OP's data. November looks more like a recovery month compared to a very bad September and bad October. This is very, very disheartening. Finishing my degree in exactly 2 weeks. Don't wait until you get the degree to start applying! Send out those applications in droves until you start to get tired of it. Then study for tests. When you get sick of studying, get back to applying. It's honestly just a numbers game. My experience is that the degree doesn't matter that much if you have experience. So if you have no experience yet it might be tougher. Hiring will probably pick up again next year, though. I hope you find something. Then good luck with that. I remember 2009 which was also a hard year but it wasn't impossible. Maybe start right after you are done. I'm friends of friends of a couple 2002 CS grads, which was much worse for tech than 2009. All of them ended up working outside of tech. It's really hard to work for a couple of years doing non-tech stuff and then try getting a new grad job when you're no longer a new grad. My recommendation is a Master's degree if you really want to stay in tech. I graduated in 2002 with a BSCS. Went into the Army for 4 years, and was able to get an entry level CS job afterwards with a defense contractor (left the Army with a clearance). Though, at that point, it was actually a pretty significant pay cut from my Army pay (I was an O-3 by the time I got out). Dot-bomb was winter in the US. I quickly lucked into a position with someone I knew, not a developer role but in tech. Were still some very rough times and my comp wasn't great for the better part of that decade. But things could easily have been much worse. I graduated in Nov 2001, and had to do non-tech for almost a year. After that it was small intermittent contracts for another year. Had an interview a couple weeks ago which, unfortunately, didn't work out. Will do. -- remember - it's a sales game - learn about how a sales funnel works - apply to your job search - X applications - turn into Y interviews for Z job -- Yep. I start a new job on Monday. Job hunting turns into a funnel quick - slow at the beginning, so you apply to everything - but at a certain point you start getting overwhelmed with interviews and you become more discriminating about what you allow into the beginning of the pipe. Important! If it is offline, please refer to this image until I get it back online: https://gcdnb.pbrd.co/images/ElMG6yee19KT.png?o=1 Thank you. I'd hate to make a joke-in-poor-taste about the German Job Market Dashboard crashing. I would take no offence, the dashboard is implemented with https://dash.plotly.com/ which is a great tool for prototyping but doesn't seem to handle concurrent users very well. As the Dash frontend wasn't able to handle the traffic I have replaced the dynamic dashboard with static HTML, sorry for that. I will replace it with the real dashboard when I am able to handle the traffic. OP's disclaimer: I started this hobby project one year ago. The point of this project was and still is to learn about new technologies. Maybe I am lacking the statistical background, or the data since it goes back only 13 months, to make such an statement but when I starting seeing these trends in the number of job posts, I thought that it would be interested to share it in the Hacker News community and get some feedback. About whether these trends that we see in the graphs are something real or just a data issue like seasonality or a problem with the data sources, I think we will soon know from the mainstream media. As always the time will tell. Regardless of the final (statistical) outcome, I think the architecture of your project is impressive. Kudos! The presentation is "nice", but not even close to what it needs to be for making any assumptions on the result. Questions that are necessary to be clarified: 1) What is your source? - the slides only state it's the biggest platform.
2) How do you compensate, that your dataset only represents x% of the real offers?
3) If you think you have a reasonable share of the job offers, how do you justify this assumption? These are the main questions, that I would ask if you tell me to use it as business information - all the technical stuff is secondary. But there are more: 4) How do you compensate a change in recruitment strategy? You are just looking at one platform.
5) What makes you sure, that every item in your dataset is interpreted correctly? Can you somehow make sure, that your parsing and reformating is always correct? Can you verify that? Can you detect if not?
6) Can you detect a change on the website itself, e.g. new categories, differences in job description? Also my experience is different, with getting a ton of job offers currently, directly via business network, I'm willing to look at a statistics based bigger picture that changes my mind. But the interpretation and the harsh change, together with these open questions make me believe that you more likely have a technical problem. So my advise would be, spend a lot more slides on the data and not the acquisition, because all acquisition effort is useless if you can not verify your data and results. > https://petra.carrion.io/job-market-analytics/slides/#/6/0/0 Is there some German/EU law that requires job postings to be available for at minimum one calendar month? Cause that spike at ~30-31 days is real conspicuous. Or at HR people just defaulting to 30/60/90 days for the posting being automatically removed? I would assume that it's not due to HR choosing to take them offline, but because that's the interval you pay for them on the platform that was scraped. Not sure which sources she crawled ... Many of the German job boards require companies to purchase job ads which expire after 30 days. I cannot say if it is a law or the pricing plan but 30 days seems to be the most recurrent job post length. That means that the crash probably started in October and we started seeing it on the data at the beginning of November. Looking at it again, I don't think the law idea I put forth makes sense. The pricing plan sounds more likely, especially given the ringing behaviour at multiples of 30/31 days. I guess a large majority of those postings go unfilled. Do you do any work to try to find if a position has been re-posted and recombine it with its previous posting for the purpose of this calculation? I don't know if one can draw the October conclusion. Would HR leave the posting up to expire on their own if they know they won't hire anyone? I say that since they seem to remove them (presumably because they got enough applicants or actually hired someone) before the 30/31 day mark. Interesting! Are you planning to keep recollecting data for the year 2023? > the major job offer portal in Germany for over one year I hope that's Linkedin. Because if it's another site like Monster, StepStone or similars, I'm afraid the data is probably garbage. I don't know a single engineer in Germany who uses anything but Linkedin (well, many engineers use niche job boards like remoteok, or directly use the company's career section... but no single one uses Monster/StepStone/Glassdoor, because they are total crap) Edit: the data source seems to be stepstone.de. That's unfortunate. Is there any chance to run this against Linkedin for 2023? That would be awesome! (https://github.com/petracarrion/job-market-analytics/commit/...) seems stepstone is the only source... perhaps try to compare against "agentur für arbeit", see https://github.com/bundesAPI/jobsuche-api (there are multiple issues with the api - some parts/parameters are broken....) I am working on integrating additional sources, and Linkedin will be definitely one of them. In my area, COVID stats are updated by the local public health agency daily, but often deaths are only included in the death data a month after the death occurs. So the COVID death data is good only for dates that are about a month old. Is there any thing similar going on here? Is the data for the past few weeks of the plot [1] as reliable as the data in July? After some research, my hypothesis might be wrong. Comparing the different data sources I have found out that there is a drop only in one them, so my best guest is that an increase in the job post price or a change in the business model (e.g. shortening the length of the time a job is posted) is the most likely reason for the drop we can see in November. Exploding energy costs will do that. Very nice project and presentation deck - well done ! That being said, with the data you have available here, you can’t conclude anything at all - you would need several years of data to rule out seasonalities, and also take into account some external factors influencing it. Ps. Are you sure you’re not using something like rolling average and you just don’t have the data points (if it’s centered around current date) ? This is a link to the latest statement from the labor agency in germany. Due to seasonal effects, employment increased, of course this is only a short statement and does not include details. Statement is in german. Is this the year end factor ? Generally there is less recruitment towards end of the year (On top of the generally gloomy market) I don't think so, note how last year, there was still a small job post increase from middle of November until end of December. My assumption is that we are moving back to a pre-covid buyers market for the tech industry. It did feel like childs play to get a job during covid, from anywhere. I am certainly happy that I found (German) clients and a job during the covid period, and don't have to go looking for them now. If everyone who found out the "German job market is crashing", "German economy is crashing" or "the EU economy is crashing" would have sent me 1 EUR, I wouldn't need the German job market any longer. Guess we're safe. I can also see this by the messages from recruiters I get, really dropped off in the last weeks This is a great project! But I would caution against being quick to interpret these numbers until you actually accumulated many years of data and are able to compare it to other macro economic variables such as the unemployment rate or changes in wages. Right now, you only have a few months of data, and it might be the case that this just happens every few months and doesn't mean much. That's the point of time series -- you need to accumulate them over long periods of time and then you start processing them to understand them better to see if they are leading/contemporary/trailing predictors of what you really care about -- in this case, the tech job market. So keep it up! But also wait a bit before using it :) In the future I wonder what other types of news like this will spread by way of data collected via someone’s pet/side projects. I guess a recession will do that? Would love to see other countries' breakdowns. I'm going to flag this for IMO sensationalist interpretation of data... It’s currently offline It seems that Dash cannot take the Hacker News traffic, here is a screenshot until it is online again: https://gcdnb.pbrd.co/images/ElMG6yee19KT.png?o=1 does it go back any further than end of 2021? Unfortunately, I have data only from October 2021, that's why the longest time filter is 12 months. are there any other sources for this?
