Show HN: HiSHtory: Your shell history in context, synced, and queryable
github.comhiSHtory is a better shell history. It stores your shell history in context (what directory you ran the command it, whether it succeeded or failed, how long it took, etc). This is all stored locally and end-to-end encrypted for syncing to to all your other computers. All of this is easily queryable via Control-R and via the hishtory CLI. This means from your laptop, you can easily find that complex bash pipeline you wrote on your server, and see the context in which you ran it. Sending your command history automatically over the network is always an huge security risk. This tool atleast have a security section in the readme, but nothing about operational security and how to guard against hacks of one synced agent. I will try it because the history tool looks like an interesting take, but the security model is not ok for multi computer use. I think my main concern here is the fact that you're putting secrets in your shell history! Mistake happen. Yesterday I was tired and working on a friend serveur. The ssh prompt was different and I entered the password from a account while the remote port has hang up on me. So effectively typing “password” straight on my shell. His password is in my history now. I removed it but that type of thing happen all the time. (
Not that much in a professional settings, password tends to fade away ) Pro-tip: if you wanna hide stuff from your shell history, enter a space/blank before the actual command. Not that it probably would have helped in this situation, but maybe useful for others. Usually if I have to enter secrets in my shell, I do something like this in order to save the useful stuff to history but hiding the secrets: - ` export PASSWORD=hunter2` (notice the space before "export") - `service-cli login username $PASSWORD` (no space before, gets saved to shell history) While ignoring commands preceded by a space character is absolutely one of my very favorite shell features, it usually isn't enabled by default (YMMV). HISTCONTROL=ignorespace # bash setopt hist_ignore_space # zsh These go into ~/.bashrc or ~/.zshrc, respectively. zsh is really smart about this and always enables you to retrieve the immediate previous line with arrow-up, no matter whether it's got preceding whitespace or not. +1 to prefixing with a space. hiSHtory supports this out of the box so that anything prefixed with a space isn't recorded (even if your shell doesn't do that by default!). And if you do ever mess up, `hishtory redact` can be used to delete history entries. Other options: - read -s PASSWORD - service-cli login username $(cat) At least you probably don't want to use `export` when setting the variable. why use `export` ?
That is a terrible idea. I've certainly entered passwords at the shell prompt by accident. This doesn't happen often, but it sure does. Some apps only accept password in commandline. The better written ones might accept it in env but that will also stay in history. But if someone have access to your home they can put keylogger in the .profile/.bashrc so it's not really relevant... Sure, but something like this also works well: `the-command --password $(read -s;echo $REPLY)` yeah now instead of history you can get the app's password via `ps aux` :D also you still have the `export REPLY=password` in history. sure you can source it out of file but, well, that file have same permissions (or looser! if you forgot to change) as your .bash_history file. `cat password|command` is the safest one (or, well, something 12 factor app clowns hate, a config file) How does this compare to Atuin [0]? [0]: https://atuin.sh/ Atuin* also allows for opt-in sync, easily hosting your own server, and switching search context for your shell history (ie whether you want to search the current directory, session, host, or all history on all hosts). We also have a bunch of different search modes that query history differently (fuzzy, prefix, etc) But otherwise yes, the tools are pretty similar! Nice to see another in the space *: context, I'm the maintainer so may be biased ;) Is there an arm64 version of the docker image for the atuin server? I'd really like to be able to deploy this to my microk8s cluster. I know it's a challenge, but, please, add deletion! Thanks! First part of deletion went out in v12, we should wrap up the rest in v13! Homebrew just released v12 today! Yay! Hmmm, Homebrew still has v11. Ah, I hadn't come across Atuin before! They're pretty similar, but I think one significant advantage of hiSHtory is the "Custom Columns" support where you can populate custom columns with the output of an arbitrary shell script. For example, you can use this to collect the current git branch as an additional column. Just tried this, and it appears to cause a very significant lag (~2 sec) after each shell command. Removing the sourcing of the config from bashrc and opening a new shell immediately gets rid of the lag. Maybe it's because I have HISTSIZE set to a large value... Ah, thank you commenting on this! This is absolutely unintentional and was the fault of a missing comment in the bash script (that I didn't notice because I generally use zsh). See https://github.com/ddworken/hishtory/commit/72ff95ab8b23c3be... and if you run `hishtory update` it should be all fixed. So, this syncs through a hosted server somewhere? Can I host my own? That information should be front and center imo, I didn't see in anywhere in the Readme. I also did not like that it's hidden away in the code. It is easy to self-host it though. https://github.com/ddworken/hishtory/issues/7 I also did not enjoy that the install script is not in the repo. Actually the install script is in the repo! See https://github.com/ddworken/hishtory/tree/master/backend/web... How does this differ from fzf, peco or autin? A table comparing these and existing tools would be ideal to show what's unique about this one. Is my new Mycroft AI device capturing my thoughts and sending them on to a Borg-like hivemind? I was just thinking of something like this the other day and here it is on HN! All jokes aside, from reading the threads, I see there are other alternatives, most interestingly atuin and fzf. I used fzf for a while, but didn't use the history features. Atuin looks amazing, and installs using nix-shell. My litmus test for whether something is safe to use is nix-shell these days; I'm not excited about running a curl script into python as my starting point! Does anyone have references on whether atuin and fzf can be used together, or should you only use one or the other? I liked fzf but the encrypted remote sync feature of atuin looks really fascinating. Looks cool. Is it possible to disable sync? For security reasons, I would rather keep everything local I didn't investigate how to run the backend server (is its address configurable?) but if it can be self hosted it could be run locally. The advantage is that VMs and maybe even containers can send their history to the local server. If it can't be self hosted, despite end to end encryption it's a big no no for me. Edit. I read the code Same. This tool would've been much better if it didn't reinvent syncing, functioned entirely offline, and just focused on the core problem of contextual shell history. There are plenty of syncing tools that handle security well, and the author could've recommended some that integrate well with this project. As it is, this is a blocker for me even trying it out. Even if it could eventually be disabled, I don't want it to have network access at all. See https://github.com/ddworken/hishtory/issues/4 This isn't supported quite yet, but is the next item on the road map. Though keep in mind that all history entries are end to end encrypted, so even with syncing enabled, your shell history isn't accessible to the backend. It would still reassure some people who are skittish about that sort of thing; I can think of people I would otherwise recommend it to who would would automatically say no if it syncs. Agreed! I'll make sure to get this implemented ASAP. :) I use syncthing instead. But if I switch computers quickly there are conflicts which I resolve like this: I didn't see the install.py script in the repo. Would be a good idea to include it. What was much more interesting to me is how this tool hooks into the shell to record commands. This can be found here: https://github.com/ddworken/hishtory/tree/master/client/lib/... See https://github.com/ddworken/hishtory/blob/master/backend/web... Also, Github pro tip: Press `t` when on any github repo to bring up fuzzy matching that can be used to easily find any specific file. Does it handle multiple writers on the same machine?
For example two people ssh-ing with the same user to a remote machine with hishtory Yes! Though they'll have a single shared history log since there is no way to distinguish between them. Looks nice! Do all of these shell history replacements completely override native history functionality or just add on to it? (and if I delete something using `hishtory redact` does it delete it from my native shell history too?) Also just a heads up, I find the demo gif to be unintelligibly fast. It just adds to it, your standard shell history is still there and completely unaffected by this tool. And if you prefer using your native shell's control-r you can also do that. > if I delete something using `hishtory redact` does it delete it from my native shell history too It does not (though if you think it should, please open an issue and I'm happy to add this!) > Also just a heads up, I find the demo gif to be unintelligibly fast. Ah thanks! Updating. Hi,
Sometimes things slips into shell-history, it’s convenient to be able to redact it. Is the history visible and editable by the user ? ( does not have to be overriding a existing command ) Yes! See the `hishtory redact` command to do exactly this. I wrote something similar that I use across 2 laptops, so it stores it in AWS. The gist of it is firehose to S3 then use Athena to query. My zsh shell sources a script that has functions that uses the AWS cli to post data or query. Out of curiosity: why not straight to s3? I'm confused why you need python 3 to install a Go app? > curl https://hishtory.dev/install.py | python3 - If you'd rather `go install` it, that works too! The advantage of this is that it doesn't require users to have go installed. And if you look at the script, all it does is call an API to determine the latest pre-built version for the current OS and install it. Anyone/anything solving the issue of bash histories in multiple (intricated) tmux sessions is an INSTANT win !!!! [Q: can this tool help? As far as in understand, it seems that YEEEES!!!] I solved that particular issue by using this in my .bashrc #write to history file at each shell prompt Yup exactly! This way you'll never again lose a history entry. directory context is quite an argument, I assume I can jump back into the directory with the line both - not just get the cli+args? (Btw I read the util name in Sean Connerys voice) Yup exactly! And you can even collect other custom metadata to make it easier to jump write back to where you were. An uninstall feature would be useful. See https://github.com/ddworken/hishtory/issues/6 for info on how to disable control-R and how to uninstall. And sorry about this, I agree that a cleaner way to uninstall is needed and will get this implemented soon! cool. nice interface! another suggestion is to have a setting for the timestamp format Is it doing anything clever for the search? Based on path context etc? david is one of the smartest people I've ever worked with at Google. It's good to see him here :) I have been using Atuin for sometime, but I must say, though that looks cool, the fuzzy search results out-of-the-box are useless to me. I used to use something before, probably McFly, that had way better search results. > atuin > You may use either the server I host, or host your own! Or just don't use sync at all. I sincerely appreciate the fact that you provided these as alternatives, because wow, offline only, with no synchronization? That's honestly really nice to see compared to this tool. I've been using Resh[0] for the past 6 months or so. A rich and queryable shell history is a massive boost in day-to-day productivity. The syncing described here is a pretty cool feature. In addition, hiSHtory also supports fish for anyone who uses fish! There's an app for Macs with iCloud sync as well: https://loshadki.app/shellhistory/ This looks great. Thank you! If Atuin or HiSHtory would only let me sync through iCloud or something else like Synching. Ah very cool! One advantage of hiSHtory over this is that hiSHtory also supports Linux (since it isn't bound to iCloud sync). I do something similar but much, much simpler with a 4-line `precmd` [1] in my .zshrc file. It just records timestamp, directory and command to a file per day. Then you can use standard grep & standard tools to find things. [1] https://til.marcuse.info/bash/bash-infinite-history.html Same but with bash, and sqlite on the backend, fzy on the frontend: https://github.com/csdvrx/bash-timestamping-sqlite This is the default on ohmyzsh :) This is interesting but i'm not interested in the sharing of the history, and i'm not sure what this offers that using fzf with zsh doesn't
func getServerHostname() string {
if server := os.Getenv("HISHTORY_SERVER"); server != "" {
return server
}
return "https://api.hishtory.dev"
}
function sync_history
cat ~/.local/share/fish/fish_history.sync-conflict-* >>~/.local/share/fish/fish_history
rm ~/.local/share/fish/fish_history.sync-conflict-*
end
(Unless I read your comment wrong.) export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"