Syncweb
An offline-first distributed web
This ‘World Wide Web’ was just a lame text format and a lot of connected directories.
Ted Nelson
Syncweb is not a physical network, but rather a logical network resting on top of several existing physical networks. These networks include, but are not limited to, Syncthing nodes and relays, NNCP, the Internet, an Ethernet. What is important is that two neighboring systems on Syncweb have some method to exchange files, from one system to the other, and once on the receiving system, processed by the Block Exchange Protocol software on that system.
Install
Install syncweb-automatic (optional)
Syncweb-automatic is an optional daemon that will auto-accept new local devices and their folders
curl -s https://raw.githubusercontent.com/chapmanjacobd/syncweb-py/refs/heads/main/examples/install.sh | bashUsage
Start your syncweb cluster by creating your first folder (can be an existing folder)
$ syncweb create ./audio/ sync://audio#CKTVWGQ-XBRFFRH-YTRPQ5G-YDA5YXI-N66GA5J-XVBGEZ3-PD56G6Y-N7TEAQC
Share your new syncweb folder with someone else by having them run:
$ syncweb join sync://audio#CKTVWGQ-XBRFFRH-YTRPQ5G-YDA5YXI-N66GA5J-XVBGEZ3-PD56G6Y-N7TEAQC Local Device ID: NXL7XBL-VPNDOSR-QXU7WI7-NEUI65A-TWN7YGT-WS2U457-NTZNGB4-J6IYDQH
You will then need to accept their device ID. Their device ID is printed for convenience when running the previous command syncweb join but they can also find their 🏠 device ID by running syncweb devices.
$ syncweb accept --folders=audio NXL7XBL-VPNDOSR-QXU7WI7-NEUI65A-TWN7YGT-WS2U457-NTZNGB4-J6IYDQH
Now you're ready to share files!
List files
$ syncweb ls --long audio/ Type Size Modified Name ----------------------------------- d 2.7GiB 06 Oct 20:56 Recordings/ d 3.2MiB 28 Jul 2022 Documentation/
Find files
$ syncweb find --type=f --ext MKA --size=-20M --min-depth=2 Test
audio/Recordings/TestRecording_1.mka
audio/Recordings/TestRecording_22.mka
audio/Recordings/TestRecording_23.mka
...
# shorthand
$ syncweb find -tf -eMKA -S=-20M -d=+2 TestSort
$ syncweb find -tf -eMKA -S=-20M -d=+2 Test | syncweb sort "balanced,frecency" | tee download_list.txt audio/Recordings/TestRecording_23.mka audio/Recordings/TestRecording_22.mka audio/Recordings/TestRecording_1.mka
Download
$ cat download_list.txt | syncweb download --yes Download Summary: --------------------------------------------------------------------------------------- Folder ID Files Total Size Usable Pending Buffer Shared Status --------------------------------------------------------------------------------------- audio 87 216.9MiB 114.6GiB - 18.4GiB (1%) No OK --------------------------------------------------------------------------------------- TOTAL 87 216.9MiB --------------------------------------------------------------------------------------- Mark 87 files (216.9MiB) for download? [y/N]:
List devices
$ syncweb devices Device ID Name Last Seen Duration Bandwidth Limit --------------------------------------------------------------- ------- --------------------- ---------- ----------------- FXVNMWB-RHVAHJX-GG6QRDK-BMBO7KH-526SY6Y-6YVKEIE-2PPM5P7-F62C2AT syncweb 🏠 Unlimited 7ZEWMJV-3JBF4CI-F4Z4CWC-U6232BS-3CP6BUI-FSXZORS-JHGJ4V3-5QDWCAE syncweb 😴 23 days ago, 19:34 1.3 days Unlimited
List folders
syncweb folders Folder ID Label Path Local Needed Global Free Sync Status Peers Errors ----------- ------- --------------------------- ---------------- ----------------- -------------------- ------ ------------- ------- ------------------- content - /tmp/tmp.WwxPwoouIa/content 0 files (0Bytes) 1 files (10.8GiB) 753 files (170.5GiB) - 94% error 1 folder path missing
Debugging
You can start another instance of Syncweb like this:
syncweb --home=/tmp/1/ join sync://test#CKTVWGQ-XBRFFRH-YTRPQ5G-YDA5YXI-N66GA5J-XVBGEZ3-PD56G6Y-N7TEAQC syncweb --home=/tmp/1/ repl
Future Aspirations
What Syncweb is
Syncweb builds on top of Syncthing as an opinionated selective-sync configuration helper.
The advantages to using it are that it is offline first. You can download a whole website and use the site fully offline. When you come back online the new changes and updates will be synced and your comments and interactions will be automatically uploaded. It is delay-tolerant.
The disadvantage is that browser support for Syncweb URLs is virtually non-existant at this time. I have no plans to work on this aspect. Feel free to lead the charge!
The other really big disadvantage is that Syncweb is fragmented. But this limitation encourages small, productive, file-sharing groups! See what other people are sharing and find a group that matches your interests.
What Syncweb is not
Syncweb will never replace your online banking app. While it may be possible to write something equivalent, I imagine doing so will be very clunky. The traditional web has very mature patterns for building. Requests are atomically mapped out across multiple services.
The traditional web has a robust line of authority via the Domain Name System so you can easily know whether you are on your bank's website or not. Syncweb has left this authority up to the community. Syncthing does not have a built-in certificate revocation mechanism like Certificate Authorities (CAs) do. You control the trust relationships of your devices directly.
Links
For example, a browser can be used in AFS by using “file://” rather than “http://” in addresses. All of the powerful caching and consistence-maintenance machinery that is built into AFS would then have been accessible through a user-friendly tool that has eventually proved to be enormously valuable. It is possible that the browser and AFS could have had a much more symbiotic evolution, as HTTP and browsers eventually did.
Mahadev Satyanarayanan
_____________
'-------------.`-.
/..---..--.\\ `._________________________________________
//|| || \\\ `-\\-----\\-----\\-----\\-----\\-----\\--\
__.'/ || || \\\ \\ \\ \\ \\ \\ \\ \
/ /__||___||___.' \\ \\ \\ \\ \\ \\ \\ |
| | -| \\ \\ \\ \\ \\ \\ \\/
| |___|________ \\ \\ \\ \\ \\ \\_.-'
[ ____ /.-----------.\ \\ \\ \\ \\ \\ .'
| |____|/ .-'''''''-. \\ \\ \\ \\ .-'''''''-.\\_/
| |____|.' '.\\ \\____....----.' '.
| |___ / .-----. \\\______....---/ .-----. \
| |___| / o o o \ \|============| / o o o \ \
| |__ | | o o | ||____________| | o o | |
[_.|___\ \ o o o / | LGB\ \ o o o / |
. . \ '-----' / . .. . . \ '-----' / . .
. . . '. .' . . . . '. .' . .
.. . . '-._ _ _.-' . . . . . '-._ _ _.-' . .
UNDER CONSTRUCTION