Category: Software Engineering

Converting HTML entities to UTF-8 in VimScript

I wrote a small script to fetch a web page title, inserting it as amarkdown formatted link: [title](URL) One of of the issues I ran into was that some titles contain HTML entities and Vim has no built-in way to deal with this.

The solution I came up with wasn’t that grand, but the interesting part was that I learned about a UNIX utility, that also ships with MacOS, called textutil.

The solution was to do a system call passing a string with HTML entities, getting back a UTF-8 formatted string that VIM understands. Here is example that you can inVim:

let oldtitle ='&#128075; hello &lt; world &#x1f310; <   &#60 It&#8217;s a great day.'
let newtitle = system( 'echo ' . shellescape(oldtitle) . ' | textutil -convert txt -format html -stdin -stdout')
echo newtitle
Example output.

The special part I want to point your attention to is ' | textutil -convert txt -format html -stdin -stdout'. This pipes the title into the textutil command which does all the conversions for us.

You can find the full function below.

function! AddLink()
	let url = input('URL to add? ')
	if empty(url)
		return
	endif
	let html = system('curl -s ' . shellescape(url))
	let regex = '\c.*head.*<title[^>]*>\_s*\zs.\{-}\ze\_s*<\/title>'
	let title = substitute(matchstr(html, regex), "\n", ' ', 'g')
	let title = system( 'echo ' . shellescape(title) . ' | textutil -convert txt -format html -stdin -stdout')
	if empty(title)
		let title = 'Unknown'
	endif
	put ='[' . title . '](' . url . ')'
endfunction

Remote work success

Remote worker, like a lamp in the darkness

The global pandemic induced a mass workplace migration. Many office inhabitants were thrown into the wild remote working paradise. The expected decrease in Covid-19 case numbers will see most people abandon the remote working paradise, but for some, the distributed work environment is here to stay. I’m writing this guide for those who will become permanent remote workers.

I have worked remotely for over 7 years and still find both glorious and challenging. Along with flexibility and control, remote work requires great discipline and support.

Over the next few weeks I’ll be posting on the pillars of remote work success. I’ll touch on inner circle support, time boundaries, self care and much more. If you are interested in learning more about remote work and other related topics please subscribe below.

This is the list of topics I’ll blog about in the coming weeks. I’ll update the titles with links to the posts as I publish them.

1 – Success with your support network

Update: After thinking through these topic, I realised that I don’t have much more to say beyond the titles. So here they are. Consider them and make sure you do some of these things.

2 – Time boundaries

3 – Find a dedicated space

4 – Remove Distractions

5 – Structure every day

6 – Take great breaks.

7 – Get out and away for remote work success.

8 – Work hard on yourself .

9 – Look after yourself.

These topics are the pillars that have helped me over the years and I’m sure it will guide you in figuring out how to succeed in a remote working environment.

Software Complexity

It is easy to define software complexity, but not so easy to define how complex a specific piece of software is. There have been lots of work in academia to find ways to describe it, but these approaches are not generally applied. Complex software directly refers to its effects on the human mind.

The software industry has a few methods for evaluating complexity. A well-known method is Cyclomatic Complexity, by Thomas J. McCabe, Sr. Cyclomatic complexity plots the number of linearly independent possible paths through a program. The reason why more possible paths introduce complexity is this; when changing a program, the maintainer needs to understand and keep those paths in mind to avoid introducing new bugs. Our human brains have limited capacity for the number of options we can consider at the same time.

Along with looking at the actual program, you must also consider the context. One must weigh in the problem it attempts to solve, the domain it targets, and a few other factors. In some cases, you can simply glance and call it complex, but for others, complexity is revealed as you dig deeper.

All software starts as a set of requirements, and this initial phase is often where we sow seeds of complexity. Software Requirements are usually initiated by people who know what they want, but not necessarily how to build it. This may result in oversimplification of the efforts required, which may lead to shortcuts resulting in unwanted complexity. The requirements phase is the first place to root out complexity.

One can’t always blame those who create these requirements, as they are often at the mercy of a complex problem domain. Think about aerospace or naval technology as examples. Certain fields are by nature complex and thus making the systems they produce have a higher level of necessary complexity.

There are two complexity flavours, according to academia. The first one, obviously named, is Essential complexity. It is a level of complexity that is absolutely required for the system to function. The next one, also accurately named, is avoidable complexity. Though these names are clear, figuring out which parts are avoidable takes a lot of time. Software needs to be taken on the road and through all its paces. We’ve seen people use software in ways the designers haven’t thought about forcing one to rethink avoidable and essential. Essential to whom? Avoidable for whom?

Simple processes can produce complex outcomes. Only the user experiences complexity. Why are we writing software? Is it for Business, business systems, end-users, or the next maintainer?

Attributing levels of complexity is a very subjective act. You should take into account all dimensions involved in a systems life cycle.

It is easy to think that complexity is a problem, but as long as a piece of software addresses a set purpose, the complexity is secondary. If the goal is for it to be simple, complexity is primary.

PHP is just fine

hammers in a row

A programming language starts decaying right after inception. The idea that a language is perfect soon hits the harsh reality of users running into situations the author(s) have not anticipated.

The law of the instrument mentions the following statement by Abraham Maslow:

I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.

It takes a long time to learn how to use a tool effectively. This leads programmers to use a familiar language when they are confronted with building something new. I think this is true for all tools. One of my hammers, the PHP language, has been around since 1994 and it is being used by 78% of all websites (source: W3Techs). PHP is showing its age and the idea of it being replaced by something new and fresh has been gaining momentum for a long time.

When you look at any programming language, given enough time to mature, you will notice the question of retirement come up. You can see how this is just a google search away: “Is this language dead?“. I found questions about all popular programming languages.

The concept of dead languages reminds me of the language I had to learn as a student, COBOL. It was created around 1960, and now it so old that modern technologists have completely forgotten about it. No-one is raving about how amazing it is, no new courses are popularly teaching it and most people just want it to go away, but in the midst of all this, COBOL is still alive and is being used by many financial institutions. COBOL solves a very real business problem and so does PHP.

The simplicity that PHP affords those who learn it and the ease of getting from 0 to 1 makes it a heavyweight in the website programming languages league. It allows people to test out ideas faster and cheaper than any alternative out there.

This old programming language is just fine for building websites and web-based solutions. It can be the final solution or act as a prototyping tool. It is a gateway to the world of programming. A way to think about problems and address the needs of your users.

Relevancy has everything to do with the problems you solve and nothing to do with the tools you use. If you solve relevant problems with ancient tools, you’ve still solved a real problem. Yes, the tools can make certain classes of problems easier to deal with but ease is relevant. PHP was my gateway into the world of programming and a way to understanding and learn other languages.

This old language is fine for solving problems, great at being used as a hammer for most nails and great as a teaching tool while learning about its weakness and where it may not be the best tool for the job.

The main thing I take away from all the language conversations is this. We have customers and clients who need our skills to solve problems. If the Hammer works use it, if not find a working tool quickly and solve the problem and don’t allow perfection to stand in the way of good enough.

Choose boring technology

I read an interesting article about choosing boring technology. Here are my key take a ways:

  • New shiny tools have this one big problem. They have more unknowns than tools that have been around for a while.
  • Boring is anything that has been around for a long time that works and have been tested at scale. Examples: MySql, PHP, Python. Boring is not bad.
  • Boring technology may also be newer but it is the default stack in any organisation.
  • Boring can save you time and money. Boring should be recommended.

Read more at https://mcfunley.com/choose-boring-technology

WooCommerce Payments now in Beta

I’m excited to share that our team has just released the beta version of a Payments service powered by WooCommerce, WooCommerce Payments.

Users of this gateway will be able to manage all payment related tasks, without leaving the the WooCommerce admin interface. We hope to roll out more features and support for more countries in the future, but for now we celebrate this milestone.

It was really great to join an amazing team as we wrapped up the final bits of this product, Kudos to all involved.

Now the real work begins and I’m excited to see the types of businesses that will benefit from this experience,

Take a look at all it can do for you here: https://woocommerce.com/payments/

The plugin is now available on the WordPress.org repository: https://wordpress.org/plugins/woocommerce-payments/

Building Reactive Systems: Conference Talk.

I watched an interesting talk about building high availability systems and thought the simplicity was quite fascinating.

Simple systems are those where the focus is business logic and not compute complexity. Complex business logic should not imply a complex system.

Reactive systems scale very well as they are:

  • Responsive
  • Resilient
  • Elastic
  • Message Driven

In this talk Dave Farley explains how to design these reactive systems. It’s all based around the premise of passing messages and progressing the state of domain models. With this approach components/sub systems can be decoupled, which makes it easier to reason about the system.

Note that this is not about asynchronous processes, as in call backs , but rather asynchronous design where the domain model caries the weight of process state.

Video URL: https://youtu.be/tKRa0O7aepo

Docker Compose Volumes Empty?

The problem is that you’re expecting files from the Container to be mounted on your host.
This is not the way it works: it’s the other way around:
Docker mounts your host folder in the container folder you specify. If you go inside the container, you will see that where there were supposed to be the init files, there will be nothing (or whatever was in your host folder(s)), and you can write a file in the folder and it will show up on your host.

Answer from: https://stackoverflow.com/questions/42395748/docker-compose-volume-is-empty-even-from-initialize