No Python interpreter? this simple RAT installs its own copy
isc.sans.eduCD Projekt's GoG client has a feature where users can write Python extensions [0] for third party account integrations. To enable this, they ship an entire Python interpreter alongside the app binary [1]. I guess this kind of makes sense, it is way easier than having users manage their own `venv`s and other Python environment stuff. Packaging Python apps for distribution outside of a Python development context is quite pesky.
[0]: https://github.com/gogcom/galaxy-integrations-python-api
[1]: https://support.gog.com/hc/en-us/articles/360024812473-GOG-G...
Shipping your own runtimes as part of heavy application isn't that unusual or limited to Python, especially in the Windows world.
For example, JetBrains ships its own Java runtime for all of its IDEs.
In fact, the Python world would be simplified if "ship your own runtime and env" as part of your final application was the norm. The Python runtime and stdlib by itself is like 20-30MB packaged up or something. For certain applications (especially media asset rich ones), that's not that significant.
CPython is ~7-8 MB zipped. Straight from the source: https://www.python.org/ftp/python/3.9.4/python-3.9.4-embed-a...
Omg, thanks!
I don't know how I missed that all this time... I've going off the "zero/dot" releases of https://winpython.github.io/ this whole time.
> For example, JetBrains ships its own Java runtime for all of its IDEs.
That's actually the recommended way for Java applications nowadays. There is no official current JRE for end-users to download anymore - the newest version you can download from Oracle is 8 update 281. JDK is currently at version 16.
I'm not familiar with Java programming.
Does this mean that Oracle has decided that Java 8 is the last consumer version of Java?
Or like you've said that applications shouldn't use the OS level java installation?
The latter. To ship Java applications it is recommended now that you build a custom bundle instead of relying on the system Java.
Post version 8 work was done to modularise core Java. You can now generate a stripped down custom run time to bundle with your application:
Is that just because openjdk is now the canonical (non-enterprise) jdk? Does openjdk not provide an end-user jre anymore?
I don't know the exact thought process that led to this decision but I know that people were incredibly annoyed by the "you need to update Java!" notifications and that applications were rolling their own JRE for quite some time to get around outdated JREs on customer's machines.
> In fact, the Python world would be simplified if "ship your own runtime and env" as part of your final application was the norm.
AFAIK, it is for final applications whose main purpose isn’t to be used as development tools tied to whichever python environment it is installed in.
"Omnia mea mecum porto". If it's a part of you, you can rely on it always be there when you need it.
Isn't that what PyInstaller already does? It bundles your Python app with a runtime. I've seen a few apps distributed like this, even GUI ones.
Yes, bit for malware you want your dropper to be small and lightweight so it can be downloaded and ran very quickly.
Hence and vb script that downloads python, they could have hosted the rat and downloaded that too rather than writing it line by line.
> quite pesky
I've never had a bigger packaging nightmare than for Python and at this point I've worked with a substantial spread of available languages. It's simply embarrassing, especially because I used to like recommending it to people.
The C Language thanks you for your propaganda.
C is the original language where you can reliably build an executable and ship it as a single file where it will run with no extra dependencies not already on the target system.
The C language may be complex to setup an environment. But once you have the build system complete, there are mature tools to create and distribute your artifacts. That is not the case with Python.
Python is downstream of C. Anyway you can package a C application, you can package a Python application (with some massaging).
But still, python has no mature tools for distribution? From wheels to pyinstaller to Kivy, you're just wrong.
> Anyway you can package a C application, you can package a Python application
While true, this means that packaging Python is harder than a typical C application, and requires knowledge of the C building process. That's not what people have in mind when installing Python software.
The methods you mention are exactly the reason why it is so complicated. Everyone wants to use their own method to install Python software.
There is no python distribution system that will allow me to create a binary for MacOS or Windows without running MacOS or windows, which is only one of the many problems with the existing solutions. Yes, I'm aware of the why. It just shouldn't matter to me.
That said, there's about a million packaging systems and most of them don't offer anything materially different from the others (except in weird edge cases that aren't advertised). The space is involved and annoying. It's emphatically not at all like C, which has very strong and well-documented defaults that do the things you expect (usually).
One of the main reasons I xnay any deployment of any python code to any production environments ever. It's sad and embarassing.
>xnay
Just FYI it's actually ixnay which is Pig Latin for nix, which of course means to cancel or pre-empt.
Agreed. Python is one of the worst languages in the sense of packaging and distributing code. It is incredible how bad they manage to make the whole process. And it is not for lack of effort, they have attempted several things, but the end result is complete confusion. Every year they come with a new way to solve the packaging problem, but in the end the result is just more complexity.
It won't ever be fixed until Python C API remains in it's current form. The problem aren't pure python extension, but you can't do anything sane once part of your application is not Python.
I'm sure this is part of the issue, but packaging even pure python modules is harder and more confusing than necessary. I've been saying this at least since I left for the relatively greener pastures of Node nearly a decade ago. Python fans have always had plenty of advice ready to contradict my impressions, but their advice has changed a great deal over the years. That whole time, "npm publish" has just worked, when it's even necessary at all. Node just works better.
Also, when they ship a dosbox game, the installation includes not only dosbox, but a zip file of the dosbox source code.
...and strangely - I don't know if this is only the mac version - they randomly include the manual for a PC game called Harvester
<game>/Extras/dosbox/dosbox_svn.zip ... <game>/Extras/Manual.pdf <-- Harvester manualI only own a couple of DosBox games (Duke 3D, Ultima: Worlds of Adventure 2: Martian Dreams), and whilst both include the source on Windows (dosbox-0.74-2.1.tar.gz), I can't find any manuals that look out of place.
The DMG for the Mac version of Duke 3D doesn't appear to include the source or a Harvester manual, though it seems to be based on Boxer, not just DosBox. The Mac PKG for Ultima does have the source (dosbox_svn.zip), but no (edit: out of place) manuals as far as I can tell.
I wonder if the Harvester manual you're seeing is a GoG artifact from reusing the same DosBox setup as Harvester and they forgot to remove it, as the source from Ultima includes a file called "harvester_mouse_clean.diff".
Try downloading a free game.
I found and downloaded Jill of the Jungle, which is a free game that uses dosbox and is available on mac.
after you install the game, check out
By the way, I've never seen them shipping boxer, although I have moved some games to boxer because it is much more in-tune with macos. It's really good if you have midi roms.<dir>/Jill of the Jungle The Complete Trilogy.app/Extras/Manual.pdf
Wow that's interesting! It might be a DosBox artifact and GoG didn't sanitize it. Wonder what the back story is: maybe a DosBox maintainer left it in when packaging?
DosBox is GPL. Shipping the source code isn't required, but an easy way to immediately comply with the license.
I always assumed if you ship something, that contains a GPL part, the whole product needs to be GPL too. Is it wrong, and shipping binaries is OK?
The source of whatever falls under the GPL must be made available (included with the product, download link, "write us a mail and we send you a CD", doesn't matter). Which parts of a product fall under the GPL depends on how exactly it "contains a GPL part".
E.g. When you use a patched version of Linux in your router, then you'd need to make those patches / the patched code available, but you would not need to make the code of your web GUI available.
If you are using libviralgpl.dll in your ProprietarySecrets.dll though, your entire application becomes a subject of the GPL.
It depends on whether the other parts of the product are derivative works of the GPL'd work or not.
In general, a reasonable boundary is that if it runs the GPL'd work as a command, then it's not a derivative work, but if it links against the GPL'd work, then it is. (This is of course not a distinction established in law, which knows nothing of subprocesses and linkers, but it's a good approximation, most of the time. Making a modification to a GPL'd binary to expose a particular interface for another non-GPL'd process to call probably makes that non-GPL'd process a derivative work, though. Conversely, using a standardized API that happens to be implemented by a GPL'd library probably does not make the program linking it a derivative work.)
The GPLv3 says:
> A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an “aggregate” if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate.
GOG presumably believes that they are shipping an aggregate, and DOSBox is an individual work, and the game they're shipping is a separate individual work, and the wrapper script to put them together is its own work, independent of the specific game or specific emulator. They must comply with the license of DOSBox, i.e., they must provide the corresponding sources, but that license doesn't apply to other things in the aggregate.
For comparison, the Linux kernel also says:
> NOTE! This copyright does not cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does not fall under the heading of "derived work".
That game is very strange and unnerving. I did not like it.
I've done this before with pyinstaller, and it's pretty straightforward. This wasn't even for a customer-facing application, instead an internal utility that I had initially made to save myself time. The first time I tried to distribute it without bundling the interpreter and libraries, it took a full hour to get it mostly set up for 4 people. Sure, every problem from multiple conflicting python environments, to pip commands for using a firewall's proxy, to people not having used cmd before were all solvable problems. But at the end of the day, it was more straightforward to bundle it as a .exe, give people a file path, and tell them to double-click it.
I've had both Sophos and Windows Defender pick up innocuous Pyinstaller-based packages as malware, presumably because packaging malware with Pyinstaller is such a common technique. It's quite frustrating if you intend to release the software.
Example issue: https://github.com/pyinstaller/pyinstaller/issues/3802
The other issue with using PyInstaller to build a program into a single binary is that every time the program runs, it needs to extract its own contents to a temporary directory. This causes slow start-up and, if the program exits abnormally, orphaned files left on disk.
I think other packaging tools have a workaround for this particular problem but, in my experience, have other issues. I’d love to use Python for building CLI tools, but building a binary is so fragile that I only use the language for prototyping, and rewrite in C++ (maybe Go in future?) for production.
What you are describing sounds like the onefile mode. Pyinstaller also has a onedir mode which is much faster to start. I packaged some python ml models into a pyinstaller built exe and a larger java application calls them. Onedir makes for a big install but they only take a second or two to run.
I've had similar false positives from McAfee when first installing pyinstaller, but not on the using of executables generated by pyinstaller. I hadn't known that pyinstaller was used by malware, and had assumed it was because the bundled interpreter could potentially make any system call, thus making it flag on lots of heuristics.
> assumed it was because the bundled interpreter could potentially make any system call, thus making it flag on lots of heuristics
You seriously overestimate AV “heuristics”.
Signing created binary with valid certificate usually prevents most of antivir problems, as they act way more lenient in regards to those binaries.
I've had this issue with recent versions of Pyinstaller and had to roll it back to version 3.4 to avoid it.
GOG Galaxy 2.0 is also an Electron app, so it bundles the whole Chromium as well. The whole app is over 100 MB IIRC, but nobody cares since you'll be using it to download 100 GB games anyway.
(On a side note, I highly recommend it if you play games and have them on different PC platforms - Steam, Epic, GOG, UPlay... The plugin system is unique and fantastic.)
What's the memory usage like? I don't think Electron is a good choice for a program designed to be run in parallel to demanding other programs like games.
As far as I'm aware, all launchers (Steam, Epic, Origin, etc.) use some form of a wrapped web browser. GOG Galaxy takes up around 65 MB when idle, while Steam uses almost 400 MB (yes, almost half a gig, I was surprised to see this number).
Blender includes Python for its extensions (add-ons).
And yes it can be a pain to deal with if you want to load binary modules.
You can create binaries for python using https://www.pyinstaller.org/. I've used it well for Linux and mac. Haven't used it for Windows.
Works well on Windows too, we use it with PySide2 for a game client launcher/patcher.
Pretty much every app that allows integrations in another language will bundle an interpreter. Maybe not on linux distros, but even there it can make sense due to version differences/bundled modules etc.
Sublime Text has it bundled too. I think that makes more sense than hoping that things work with whatever the system version of Python might be.
Wow, I had no idea Gog allowed that. Can you give examples of these extensions being put to good use?
I think the integrations with other platforms are done this way, so I have my games from steam, epic etc and even the titles from my xbox pc pass all in one list in GOG. Dont know about other uses for extensions.
that seems quite common in the gaming world. i've seen games packaged with tons of *.pyd files (the python dll equivalent for windows). Probably a large portion of the game logic was implemented in python
> Packaging Python apps for distribution outside of a Python development context is quite pesky.
It's actually pretty easy, as long as you skip on the various tools meant to make it an "easy one click process resulting in just one .exe" (pyinstaller, cx_Freeze, py2exe etc.).
For Linux things are a little bit different, but you're probably okay as long as you use a very old system to build it (RHEL6/7, manylinux1/2014).
Loving how it also downloads the license.txt file. Very important to respect license agreements when compromising someone's machine
I'm having a hard time deciding which DND alignment would fit that. Probably chaotic evil.
Sounds more of a lawful evil to me!
This person also probably pays their criminal taxes.
https://en.wikipedia.org/wiki/Taxation_of_illegal_income_in_...
That's how dropbox works as well, and the interpreter is a custom one that reads encrypted opcodes from the pyc files.
Yes, well, at least it's Python 3.