Holochain and CoGov Learning and Experimentation

[Geoffreyjen_Edwards]

There is a growing common interest on the Infinite Conversations/Cosmos site to explore e-governance and e-currencies, and we have had a number of fruitful exchanges with developers of Holochain and also the CoGov modules that are being built in relation to Holochain. Over the past few weeks I downloaded the development environment for Holochain to my computer and have been working my way through the tutorials in order to begin experimenting with the environment. Several others have expressed interest in doing this also. In addition, there are ongoing discussions about its relationship to other e-governance and e-currency initiatives such as blockchain and bitcoin. So I propose to use this thread to support our learning efforts but also to encourage further discussion about these issues.

I am working on a Mac - unfortunately, those on Windows computers will have different issues. However, I propose to expose some of my “teething issues” here, and try to keep separate Mac issues from Unix, Holochain and CoGov issues.

Basically, to get Holochain up and running on my computer, I went to the Holochain.org website and selected the menu item “Get Started”. They provide detailed instructions for downloading the code for each type of platform. Holochain is built inside a kind of extended programming environment called github.com - it is essentially a community of developers working across a broad range of projects of which Holochain is only one. Once github is installed, you are then ready to download the holochain code itself.

Mac issues on downloading and installing

My primary problems were all related to the fact that I have not really done any Unix work in decades. The MacOS is built on top of Unix. For most users, this fact is simply an obscure issue they pay no attention to, but it means that the Mac is also a powerful Unix computer. To access the Unix environment, you need to use a Utility app built into the Mac called “Terminal”. Terminal gives you access to a command line interpreter for Unix (where you type commands in to do stuff, instead of using the mouse).

The terminal screen looks like this :

It basically allows you to access the directory structure of your Mac using typed instructions. In Unix, of course, there are standard ways of doing this. When you first get onto Terminal and open a window, you get something like this :

It’s a bit daunting!

if you type in “pwd” the system tells you what directory you are in (present working directory, or something like that).

pwd
==> /Users/geoffreyedwards

So it tells me that I am in the Users directory in the subdirectory with my name. My computer has more than one account, hence that location.

To get stuff installed, you are going to have to be in the root directory. To get there, you use the “cd” (change directory) command. The simplest way to do this is to use, in the example I have given, “cd …” twice. “cd …” tells Unix to move up a level in the directory structure.

So

cd …
cd …
pwd
==> /

So now I am at the root level.

The instructions for holochain will install holochain, by default into a set of directories and subdirectories that it creates for you. But the instruction also tell you to make some changes to some of the root files, which is why I have shown you how to get there. In particular, you are asked to change the “.bash_profile” file. File names that begin with a “.” are normally hidden in Unix. A command such as “ls” will give you a list of all the files in the directory, but won’t always show the hidden files.

Now, to get into a file, you are going to need an editor. Fortunately, Unix has a number of built-in editors you can use. The simplest is something called “vi”, although it has been updated, and is now called “vim”. You simply type “vim” and the name of the file you either want to create or want to edit (if it is an existing file), and, presto, you are inside the editor. You need to find some documentation on vim to use it properly, however (here is a pdf manual I quite like - I find a pdf file useful in a manual since I don’t have multiple screens on my computer so having everything on line can be awkward flipping between different windows). I just hunted around on the web for a manual, my vim skills had gotten rusty!

Anyway, I made the changes asked for by the holochain installation instructions, but I did something really stupid. The two lines they want you to put into the “.bash_profile” file include quotation marks. I typed in my usual quotation marks, and went ahead with the process, and the thing stalled. Worse, you had to run “bash” to get everything working, and the program just hung - it wouldn’t take any commands or do anything at all. So I spent a considerable amount of time reading up on “shells” (bash is a Unix shell - a separate command line environment from the default one Unix uses when you first come into it, with somewhat different commands), Unix commands, what have you. I was stuck. So I emailed Ray Powell and he pointed out, when I sent him a copy of my code, that I had used “smart quotes” and not basic quotes. Of course the compiler doesn’t recognize smart quotes! Problem fixed, and everything went forward again.

As I mentioned above, running bash means that not all the Unix commands I knew worked the way I expected them to. So I hunted around on the net and found someone who had developed an extensive “.bash_profile” that provides all the right aliases I needed to work the way I expected to *see Nathaniel Landau’s page. If you are new to Unix, this probably isn’t very useful - for me it was a godsend, since without it, all my ‘reflexes’ were wrong. The rather extensive lists you see in the first Terminal image are the results of this extended .bash_profile running in the background - it lists files even when I just hit a carriage return.

Eventually, with a bit more work, you get to the point where you can code in following the tutorials. My JavaScript programming skills were also rusty, not just Unix, so I looked up a JavaScript manual and refamiliarized myself with JavaScript. You almost don’t need to do this to work through the tutorials, the language reads without much difficulty and the tutorials are well designed, but since you are going to be programming yourself eventually, it makes sense to do this.

I had some debugging to do - missing commas, etc. - all programming environments require you to check character by character what you have done, and it’s easy to omit something or make a small mistake copying.

The HoloWorld tutorial (essentially the first tutorial they propose) is quite extensive. There are some structural issues. I was reminded of a Jane Siberry song “what you do over here, you have to do over there”. You make changes to one file but you have to make appropriate and matching changes to another file. It’s okay when you are following a tutorial, but when you are developing a new programme yourself, you are going to have to keep track of these things.

One thing that took me a while to work out. The tutorials ask you to work in two types of files (with two types of extensions), “.json” and “.js” files. Why those two? I kept asking myself. Eventually I looked it up. The “.js” files are the JavaScript code files, as one might expect. The “.json” are essentially data files, that is, they contain relevant data that is used by the “.js” files - I think this is part of a principle of separating data off from code. However, JSON is a data exchange format used by a wide variety of programming languages, not just JavaScript. JSON (for JavaScript Object Notification) is constructed using JavaScript syntax, which is where the confusion arises, but it is a much more general purpose kind of file.

At the end of the HoloWorld tutorial, you are asked to run code which opens a port (“4141”). Don’t forget to do this in your browser - open a page at the address “http://localhost:4141”. I forgot to do this and the program hung again in the Terminal window. To break out of a “hung” program, press “Control-C”.

More Unix issues

By now it should be apparent you will also need a primer on Unix commands if you are working in this environment. There are lots around (here’s one that I find useful), and you can always type “help” in directly in Unix and get a list of acceptable commands.

By the time you have HoloWorld up, you now have a deep directory structure. Mine is “~/go/src/github.com/holochain/holochain-proto/testApps/holoWorld” to give you an idea. Since when you sign onto a new Terminal window you enter near the root directory, it is awkward and time consuming to type this long directory in every time (there are other tricks however that simplify the process). So I re-entered the “.bash_profile” file and added new aliases that assigned these long commands to “cdholo” and “cdhw”, so that’s all I have to type in now to get to the right directory.

I think that covers things for my first programming issues in relation to Holochain. I think one needs to work through several tutorials to really begin to understand how to program in Holochain. I will let you know how that goes.

[Geoffreyjen_Edwards]

So I ran “cointoss” which is the next tutorial proposed by the Holochain site. It bugged, and it can’t be my coding that is at fault because it installed itself. The holochain code is all beta or alpha code, however, meaning there are no guarantees it won’t crash. I’m going to try to work out the problem, because getting a random cointoss is an important function. I have thought a bit more about what I want to do as a learning process. I quickly get impatient with tutorials - I want to branch out and do something on my own.

So as a first project I am thinking of developing a Yi King app (I Ching) - I wrote one on HyperCard years ago - using the cointoss protocol if I can get it working. I’m also thinking of using the “.json” structure as a way of thinking through what kind of data we might be managing in the cosmos app (which for the time being I am calling “c_osmose” just to give it a distinct flavour from the eventual more complete “cosmos” app we will perhaps be developing). Writing code is much more daunting than just thinking through data issues is my thought.

I’m also wondering about using UML (Unified Modeling Language) - I don’t know if it is an appropriate tool for holochain. I find UML useful because it forces me to think through structural issues when organizing a programming project (usually I have students do the actual coding - can’t do that here!).

Just some early thoughts about next steps…

[madrush (Marco V Morelli)]

Wow, Geoffrey. I’m not quite ready to dive in, but it will certainly be less daunting knowing you have already taken the plunge. I’ve done a little using the Terminal utility; I’ve always wanted to be able to do more. Once I catch up with other things, I will give this a try. It would fun to try your I Ching.

My goal in learning Holochain would be to become conversant, not necessarily proficient, in the environment—so I could at least understand the implications of technical conversations. But it would be fun, if I get the time, to dive in myself and learn more. Hopefully some others will give it a try, too.

[Geoffreyjen_Edwards]

@madrush

My goal in learning Holochain would be to become conversant , not necessarily proficient, in the environment—so I could at least understand the implications of technical conversations

I’ve always taken a similar approach. I will never be a professional app developer - it neither interests me nor is it among my aptitudes. But in order to apply a new technology, one has to understand it, and this is what I feel this effort supports. Perhaps we will be able to develop some “prototypes” for Cosmos, but after that, we will need to pay someone to turn it into a robust product. My experience is that the more work you can do to prototype something like this, especially in a new area that is not well understood, the more the programmers who will eventually be doing the work will understand what they are being asked to do.

Of course, understanding the technology better also will raise new theoretical and ethical questions as well, which is an added bonus to these efforts. And sharing the challenges is also part of what we are trying to do at Cosmos…

[Geoffreyjen_Edwards]

I started to code in something for the YiKing app, but I realized that I still didn’t have a clear enough idea of the Holochain architecture to understand the steps that need to be carried out to do anything much. Of course, Holochain is so new there are no “Holochain for Idiots” manuals out yet, but I found an interesting video presentation of an earlier version of the Holochain architecture by some of the developers. The video is a bit technical, in some ways it is overly complex because the developers were still trying to get clarity on what they were doing, but the architectural principles are nonetheless present. Now, acknowledging that the exact set of functions in the current version are going to be somewhat different from the ones portrayed in the video, I have gone ahead and prepared some simplified diagrams to try to give a better sense of what is going on (that is, without wading through all 48 minutes of the video). So here is the holochain architecture, simplified in order to get the basics :

The Nucleus is where one inputs data to be stored in the holochain. This data is then stored in the chain, and then the chain is connected into the Distributed Hash Table which assigns it a hash address by which others can access the data. Hence both the data and the chain stay local to the client (user), but the hash addresses are shared so other people can access the (chained) data without corrupting it.

Each of these three basic objects has a set of functions associated with it (these will differ slightly in the current version) :

For the Nucleus, the functions are mostly data related (get, update, remove, validate, and even getLink is a kind of data). “commit” is an important function in holochain, as it involves affirming the data entry process and engages an automatic validation step. For the chain, the main functions concern the process of adding data unit (nuclei) to the chain and some validation steps for doing this. For the DHT, things get more complicated.

The diagram shown in the video is much more complex than this, because it contains many different types of objects as well as “actions” and “messages” which I have kept out of this diagram. Also, there are “top-level” functions - these are associated with the holochain environment but not with any one object within that environment. Here is a third diagram that shows these “extra” functions as well :

The holochain (top level) functions include makeNucleus() - everytime new data is to be added (and validated), a new Nucleus needs to be created, GetValidationResponse() which triggers the different validation procedures, the message Send() function which is a general purpose messaging function from what I can make out, and a function for verifying any actions carried out by the holochain.

There are several additional, ah, “elements” (I hesitate to call them objects as I don’t fully understand the implementation at this point, but I suspect they are not officially objects. Things like “DHTReceiver”, “ValidateReceiver” and “GossipReceiver” which appear to be outside the main holochain structure per se.) Also, the diagram in the video also contains actions (in red) and messages (in green). It also contains some information on internal data (e.g. entry, type, src, status, ifx, etc.) for objects, in particular the DHT object.

So the overall idea is that a program that uses holochain will need to add and validate data, then append the data onto the chain, then update the DHT and ensure that validation is adequate, and then access the data via the DHT and chains. Roughly speaking.

As for file structures, directories and subdirectories while coding. there appear to be three main directories : /dna, /ui, and /test. The “dna” directory will also contain as subdirectories the various application files. the “ui” directory will contain both “.json” and “js” files as well as “html” files as appropriate to handle the user interface. The “test” directory includes files used to monitor and track the application innards as part of a debugging effort.

[Geoffreyjen_Edwards]

I had some feedback from the folk at Holochain. On the forum, I provided the address of this thread and they have been looking at what I have been writing. They thought I did a pretty good job so far, but pointed out some additional documents that someone else on the learning curve developed - a pdf file showing additional useful diagrams about the structure of holochain apps, which was going to be my next topic of discussion anyway!

Anatomy of a Holochain App

For the purposes of discussion, I reproduce an earlier version of the first of these diagrams below :

The Schema Files are only necessary if you want to pass complex data structures (i.e. objects) to the Holochain rather than simple strings. To begin with, I suggest you do the latter. Remember in the terminology of Holochain, an “Entry” is a piece of information (data) that is appended to the chain. Zomes are a more important concept - they represent “code regions” where information (such as that stored in variables) are partially shared. Holochain “applications” will generally be subdivided into different zomes, although to begin with, one will be working on only one zome. I use the term “applications” with quotes because there is some discussion about whether “application” is the right word to use to describe Holochain code - see this very interesting article to read more about this issue. The second diagram in the document above shows that the “data traffic” within the holochain and between different holochains is called “gossip”.

Another thing I wanted to emphasize is that if one is an old hand at programming, one tends to view “validation” as that bit of ad hoc stuff you add on the end to test, debug or otherwise ensure more robustness to the code. You cannot do this in Holochain. Holochain is built on the idea that things are validated from the bottom-up - it is the whole point of Holochain. Therefore, you have to think through validation procedures when you are coding, and not relegate this to some kind of afterthought. Fortunately, the “init” procedure used to set up an empty Holochain application creates templates for the diverse validation procedures, and, in addition, the “test” environment incorporated into Holochain facilitates code that can be used to debug the development work.

Regarding learning to develop one’s own code : For long time programmers, much of what I say will seem obvious or redundant, but for people without a lot of programming experience, you need to pay attention. You will be tempted to write long sections of code and then try to compile and run the results. This is not the way to do things - you will never be able to untangle all the different problems it will create. What you need to do is write some very basic code that successfully runs (for example, by copying someone else’s code - but make sure the code is simple enough for you to understand!), and then you want to iteratively modify the code in such a way that for each change you make, you can still run the code. What I suggest is to use the procedure proposed for the HoloWorld tutorial as a rough guide to a viable programming effort. In the HoloWorld tutorial, you develop first of all a small app that just does some local things, without accessing the chain structure. Only when this code is working do you then modify the structure to allow appending “Entries” to the holochain. Furthermore, you wait until that is working before modifying the code to allow the resulting chain to be added to the DHT. This is the way to go. Try to make the smallest changes you can and ensure the code works each time. For debugging, use the test facilities. This way, when you introduce more ambitious code and it crashes (as it will!) you know that the problem comes from the latest set of changes and not, in general, from the rest of the code.

I have made progress in this sense on my YiKing application. I still have a long way to go, but I have the beginnings of a holochain structure. To begin with, I am concentrating on getting just the random number generator part working locally. Then I will use the chain and the DHT to access other players and get them to contribute to the random number generation process (using cointoss as a guide). Finally, I will work out how to display the results to the user. There is a certain satisfaction, also, in working out how to break the problem domain into manageable pieces (i.e. “functions”) that are then accessed by the different sections of code. The only real downside is that all this requires a massive investment of time!