Skip to main content

सतत

Keep reinventing

Second Brain - Archiving: Keeping resources handy

Problem Statement

We are suffering from information overloading, specially from the content behind the walled gardens of Social Media Platforms. The interfaces are designed to keep us engaged by presenting to us latest, popular, and riveting content. It is almost impossible to revisit the source or refer to such a piece of content sometime later. There are many products that are offering exactly that, “Read it later”, moving content off of these feeds and provide a focused interface to absorb things. I think following are some crucial flaws with the model of learning and consuming the content from Social Media Platforms:

  1. Consent: Non consensual customization aka optimization of the feed.
  2. Access: There is no offline mode, content is locked in.
  3. Intent: Designed to trigger a response(like, share, comment) from a user.

In rest of the post I would like to make a case that solving for “Access” would solve the remaining two problems also. When we take out the content from the platform we have the scope of rewriting the rules of engagement.

Knowledge management

As a user I am stuck in a catch 22 situation. Traditional media channels are still catching up, for any developing story their content is outdated. Social media is non stop, buzzing 24x7. How to get just the right amount of exposure and not get burned? How to regain the right of Choice? How to return to what we have read and established it as a fact in our heads? Can we reminisce our truths that are rooted in these posts?

These feeds are infinite. They are the only source of eclectic content, voices, stories, opinions, hot takes. As long as the service is up, the content would exist. We won’t get an offline experience from these services. Memory is getting cheaper everyday, but so is Internet. Social media companies won’t bother with an offline service because they are in complete control of the online experience, they have us hooked. Most importantly, offline experience doesn’t align with their business goals.

I try to keep a local reference of links and quotes from the content I read on internet in org files. It is quite an effort to manage that and maintaining the habit. I have tried to automate the process by downloading or making a note of the links I share with other people or I come across(1, 2). I will take another shot at it and I am thinking more about the problem to narrow down the scope of the project. There are many tools, products and practices to organize the knowledge in digital format. They have varying interfaces, from annotating web pages, papers, books, storing notes, wiki pages, correlate using tags, etc. I strongly feel that there is a need for not just annotating, organizing but also archiving. Archives are essential for organizing anything. And specifically: Archive all your Social Media platforms. Get your own copy of the data: posts, pictures, videos, links. Just have the dump, that way:

  1. No Big brother watching over the shoulder when you access the content. Index it, make it searchable. Tag them, highlight them, add notes, index them also, they can be searched too.
  2. No Censorship: Even if any account you follow gets blocked, deleted, you don’t loose the content.
  3. No link rot: If link shared in post is taken down, becomes private or gets blocked, you will have your own copy of it.

This tool, the Archives, should be personal. Store locally or on your own VPS, just enable users to archive the content in first place. How we process the content is a different problem. It is related and part of the bigger problem of how we consume the content. Ecosystem of plugging the archives with existing products can and will evolve.

Features:

In P.A.R.A method, a system to organize all your digital information, they talk about Archives. It is a passive collection of all the information linked to a project. In our scenario, the archive is a collection of all the information from your social media. In that sense, I think this Archive tool should have following features:

  • Local archive of all your social media feeds. From everyone you follow, archive what they share:
    • Web-pages, blogs, articles.
    • Images.
    • Audios, podcasts.
    • Videos.
  • Complete social media timelines from all your connections is accessible, available, locally. Customize, prioritize, categorize, do what ever you would like to do. Take back the control.
  • Indexed and searchable.

Existing products/methods/projects:

The list of products is every growing. Here are a few references that I found most relevant:

Thank you punchagan for your feedback and review of the post.

Striking a balance between Clobbering and Learning

Getting stuck as an "Advanced Beginner" happens. Specially in cases when we use a new tool or language to deliver a product/project. I have noticed that I approach things with a narrow mindset, I would use the tool or language to deliver what is desired. It will have expected features but its implementation won't be ideal. The process of unlearning these habit is long and often times with a deadline I end up collecting tech debt. Recently I came across some links that talked about this phenomena:

Related Conversations on Internet

There was a big thread on HackerNews around better way to learn CSS(https://news.ycombinator.com/item?id=23868355) and I found this comment relevant to my experience:

They always assume every one learned like them, by trying stuff out all of the time, until they got something working. Then they iterate from project to project, until they sorted out the bad ideas and kept the good ones. With that approach, learning CSS would probably have taken me 10 times as long.

Sure this doesn't teach you everything or makes you a pro in a week, but I always have the feeling people just cobble around for too long and should instead take at least a few days for a more structured learning approach.

Last statement of the comment struck a chord, cloberring has its limitation and it needs to be followed up with reading of fundamental concepts from a book, manual or docs.

Another post that was shared on HackerNews talks about Expert Beginner paradox: https://daedtech.com/how-developers-stop-learning-rise-of-the-expert-beginner/

There’s nothing you can do to improve as long as you keep bowling like that. You’ve maxed out. If you want to get better, you’re going to have to learn to bowl properly. You need a different ball, a different style of throwing it, and you need to put your fingers in it like a big boy. And the worst part is that you’re going to get way worse before you get better, and it will be a good bit of time before you get back to and surpass your current average.

Practices that can help with the process of clobbering and learning:

  1. Tests: unittests gives code a structure. They set basic expectations on how the code should and should not behave. If we maintain a uniform expectation through out the code base, unittests helps maintain a certain uniformity and quality.
  2. Writing documentation: For me this is like rubber duck debugging. It gives an active feedback on what are the deliverable, supported features, limitations, and upcoming features.
  3. Pairing with colleagues over the concepts and implementation. Walking through the code and explaining it to colleagues helps me identify sections of code that make me uncomfortable. Where am I weak and where should I focus to improve.
  4. Though similar to pairing, Code Reviews have their own importance and value.

These practices won't replace the need of reading Docs or Book, but they would certainly give you good quality code and keep your tech debt in check.

Clojure, hash-map, keys, keyword

tldr; Simple strings can be used as key to a hash-map. Either use get to lookup for them. Or convert them into keyword using keyword method.

hash-map are an essential Data Structures of Clojure. They support an interesting feature of keyword that can really enhance lookup experience in Clojure hash-map. 

;; Placeholder, improve it
user=> (def languages {:python "Everything is an Object."
                       :clojure "Everything is a Function."
                       :javascript "Whatever you would like it to be."})
;; To lookup in map
user=> (:python languages)
"Everything is an Object."
user=> (get languages :ruby)
nil

Syntax is easy to understand and easy to follow. So far so good. I started using it here and there. At a point I came to a situation where I had to do a lookup in a map, using a variable: 

user=> (def brands {:nike "runnin shoes"
  #_=> :spalding "basketball"
  #_=> :yonex "badminton"
  #_=> :wilson "tennis racquet"
  #_=> :kookaburra "cricket ball"})

(def brand-name "yonex")

Because we have used keyword in map brands, we can't user value stored in variable brand-name directly to do a lookup in the map. I tried silly things like :str(brand-name) (results in Execution error ) or :brand-name (returns nil ). I got confused on how to do this. Almost all examples in docs were using keyword. I tried a few things and understood that we can indeed use string as key and to fetch the value use get function: 

user=> (def brands {"nike" "runnin shoes"
  #_=> "spalding" "basketball"
  #_=> "yonex" "badminton"
  #_=> "wilson" "tennis racquet"
  #_=> "kookaburra" "cricket ball"})
#'user/brands
user=> (get brands brand-name)
"badminton"

While using keyword has simpler syntax, at times when I am using external APIs it is easier to work with string or lookup for a key in hash-map using variable. In python I do it all the time. Though I am not sure if using string as key is the recommended way.

Update <2020-07-08 Wed>

punch and I were discussing this post and he mentioned that in lisp we can use keyword as method to convert string into a keyword. After a quick search, TIL, indeed we can keyword a variable. The method converts string into a equivalent keyword: 

user=> (def brands {"nike" "runnin shoes"
  #_=> "spalding" "basketball"
  #_=> "yonex" "badminton"
  #_=> "wilson" "tennis racquet"
  #_=> "kookaburra" "cricket ball"})
#'user/brands
user=> (keyword brand-name)
:yonex
user=> ((keyword brand-name) brands)
"badminton"

Clojure Command Line Arguments II

I wrote a small blog on parsing command line earlier this week. The only comment I got on it from punch was:

I'd like to learn from this post why command-line-args didn't work. What is it, actually, the command-line-args thingy, etc.?

Those are good questions. I didn't know answer to them. As I talk in post I wanted a simple solution to parsing args and another confusing experience brought me back to the same question, "Why args behave the way they do and What are *command-line-args*". I still don't have answer to them. In this post I am documenting two things, for my own better understanding. One around reproducing the issue of jar not able to work with *command-line-args*. Second one around limited features of sequence that are supported by args

Reproducing behaviour of jar (non)handling of *command-line-args*

We create a new project using lein: 

$ lein new app cli-args
Generating a project called cli-args based on the 'app' template.
$ cd cli-args/
$ lein run
Hello, World!

We edit src/cli_args/core.clj to print args and *command-line-args* 

cat <<EOF > src/cli_args/core.clj
(ns cli-args.core
  (:gen-class))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "Printing args.." args)
  (println "Printing *command-line-args*" *command-line-args*))
EOF

$ lein run optional arguments

Now we create jar using lein uberjar 

$ lein uberjar
Compiling cli-args.core
Created target/uberjar/cli-args-0.1.0-SNAPSHOT.jar
Created target/uberjar/cli-args-0.1.0-SNAPSHOT-standalone.jar
$ cd target/uberjar/
$ java -jar cli-args-0.1.0-SNAPSHOT-standalone.jar testing more optional arguments
Printing args.. (testing more optional arguments)
Printing *command-line-args* nil

Clojure is able to handle *command-line-args* but java is not. That narrows down the problem and can possibly lead to explanation on why it is happening(Maybe in another post).

Sequence features supported by args

I noticed another anomaly with args. I was passing couple of arguments and I noticed that it doesn't support get method. 

cat <<EOF > src/cli_args/core.clj
(ns cli-args.core
  (:gen-class))

(defn -main
  "I don't do a whole lot ... yet."
  [& args]
  (println "first argument" (first args))
  (println "second argument" (second args))
  (println "third argument" (get args 3)))
EOF

This is what I noticed as I tried different inputs: 

$ lein run
first argument nil
second argument nil
third argument nil
$ lein run hello world
first argument hello
second argument world
third argument nil
$ lein run hello world 3rd argument
first argument hello
second argument world
third argument nil

The get method doesn't work. I printed type for args and it is clojure.lang.ArraySeq. For my case, I "managed" by using last and that gave me what I wanted. Still, I am running out of options and I would have to either dig deeper to understand args or fall back to using a library(tools.cli).