Help my linux server won’t reboot or halt!

In some cases it can be impossible to restart your server or just shut it down. The reasons could be many, but most times it concerns I/O blocking that could be caused by a faulty device. If you’re having the server next to you, you’ll just hit the reset button and disconnect the faulty drive. In cases when the server is not in your reach you can send a message to the kernel to HARD reboot. All commands are of course done with a root account.

Keep in mind this is the same as pressing the RESET button on a an active machine. This can corrupt data if unlucky. Dropping 1 into sysrq means that you want to access a system request. This is normally set to 0. And the trigger ‘b’ means reboot system. It’s not good practice, but it helped me to get a system back up again after hung processes eating all the IO.

Removing sensitive details from git history

In early stages of software development, it can sometimes be inviting to use a hard coded passwords for quick and dirty access. Later when the project has grown and has a more serious manner of handling authentication, you don’t want old passwords laying around in your git history. The history remains. Warning: Be sure to backup repo before trying this out!

So here is one way to replacing a password string to something else in git. An example: You have been using the password sUp3rs3cr3tpassw0rd since it’s actually a password you elsewhere you want it gone from your git history and replace by HIDDENPASS. This only works on one branch.  So a good idea is to get rid off all other branches and concentrate on the master branch.

This will go through the entire commit history of the branch replacing and rewriting the commit. Depending on the size your project this can take a very long time! It’s not a perfect solution, but it does the job. So lesson learned never use hard coded passwords in code or in a repository. A much better idea is to store it in a text file that never should be included in any commit.

If there’s a file you really want to get rid of in your commit history you can do something like this:

Although, keep in mind that all of these operations is a gateway to a world of merge conflicts for people that are syncing against the repository.

There is also another tool written in Scala that does this a lot faster, and can remove large blobs from the git history called BFG.

Source:
http://www.davidverhasselt.com/git-how-to-remove-your-password-from-a-repository/

BFG:
https://rtyley.github.io/bfg-repo-cleaner/

Search and destroy in bash

I’ve been doing a lot of web development using the Django framework lately. I’m a big fan of Python and Django since it enables you to quickly get results and keeping the code and file arrangement structure nice and tidy. When I’m trying to figure out what data I’m messing with I’m usually using the Python debugger pdb. When using the django development server and pdb it sometimes causes spawning new server instances when it restarts the server (after code changes) which prevents you to restart the server with CTRL+C.

So today I found a bash phrase which does a ps aux looking for Django instances and stopping them.

This can be applied to many other situations where things needs to be restarted or killed. Thank you Stackoverflow!

Answer on Stackoverflow:
https://stackoverflow.com/a/3510850
Django:
https://www.djangoproject.com/
ipdb:
https://pypi.python.org/pypi/ipdb/0.8.1

GhostScript magic

I’m quite used to doing presentation slides in Inkscape. However there’s no proper multi page support in Inkscape. After a quick search I found a person on Stackoverflow that had the same issue. So I found out that you can use GhostScript to merge several PDF’s into one. You can also use it in order to decrease the filesize of a PDF.

Inkscape
https://inkscape.org/en/

Stackoverflow post
http://stackoverflow.com/questions/2507766/merge-convert-multiple-pdf-files-into-one-pdf

 

 

 

Upgrading and setting up the computer for recording

This week I spent two days upgrading the computer with Windows 10 and a new SSD drive.

Step 1 – A new clean install of Windows 7

For a quite some time I’ve been wanting to entirely re-install my computer. It had gotten slow, occasionally not booting and felt unstable. Finally I got the time for some computer housekeeping.

messy_desktop

Computer’s mental state

I formatted the new drive and re-installed Windows 7. Upgrading to Windows 10 for free, requires you to possess a Windows 7 or 8 version.

I imagined that re-installing Windows 7 would be a simple and quick thing to do. But I forgot the vast amount of updates it needs to install. (Windows 7 was apparently released 2009). It took pretty much the whole night to download/install patches and service paks. It doesn’t really help that the updating procedure in Windows is painfully slow compared to Linux or OSX.

Step 2 – Upgrading 7 to 10

Windows-10-wallpapers-in-4K

Upgrading to Windows 10 went fairly quick apart from that my on-board network adapter wasn’t supported, so I needed to find a driver for it.

Windows 10 and privacy settings
Soon after the installation was complete, I had to go through all the privacy options. Windows 10 is really eager monitor your daily activities and your data in order to better suggest ads, tips and other things via Cortana. (Cortana is the “Siri” for Windows users.)
Collecting data about their users is something that Apple and Google already are doing. However it is a bit frightening that Windows 10 soon will be used daily in the public sector and governmental organisations, due to the fact that it monitors who your contacts are, who you are, what you type on your keyboard and what data you access/work with…
Is privacy for the individual officially dead? Did they win?

For privacy concerns and tips on what to disable, check this video out:

I am using the operating system in order to run my programs, code and record music. It would be a dream to have a lightweight Windows edition without all the “crayons”. I’m really not interested in a news, weather or messaging app on my desktop. I value each byte of resource the OS can give me, in order to run the programs I prefer to use. Too bad I have a hard time abandoning Windows since my audio interface at the moment only supports OSX and Windows.

After removing all the “extras”, I feel it’s actually a quite nice Windows edition (except privacy intrusion part). It feels quicker than Windows 7, and I like the new design (if you compare it to Windows 8). Even though it still takes some time to find some settings. Since laptops are more used than desktops today the battery life is important. So Windows will by default conserve power by putting disks, usb hubs on standby mode. When recording music it’s a good idea to turn these “sleeping” functions off.

The good parts of Windows 10:

  • Quick, responsive and stable
  • No metro screen
  • A fairly minimized look and feel (after disabling some elements)

The less good parts of Windows 10:

  • A lot of privacy options to disable
  • Still unknown what of your data is sent to Microsoft and parties.
  • Configuration is hidden
  • Monitors your activities

Step 3 – Installing Reaper and Superior Drummer 2

box_72dpicopy-076d9a19e671f78c6412b22a5e1b688cCockos-REAPER-4

Windows 10 automatically found my audio interface Roland Quad Capture (UA-55) and installed it. Reaper is always a graceful experience. 10Mb download and install. Done!

I recently got Superior Drummer 2 from Toontrack together with Metal Foundary SXD. It’s a massive library of different drum samples, 10 DVD’s to install. It has all the possibilities to create a great drum sound for what ever music style you want. Although it’s a bit of a double-edged sword. The more options you have, the more complex the product gets to use.

Conclusion

I think the upgrade to Windows 10 and a new SSD drive made a bit of a difference. I believe it’s healthy to once in a while do a complete re-install and try something new.

A lot is new on the surface in Windows 10 (hey the start menu is back).  However, underneath the make-up I believe there aren’t that many improvements from Windows 7. So at the moment there are no real advantages to upgrade your Windows 7 if your happy with it. I also hope that everyone knows that a solid state drive is the cheapest and most effective way to make your computer feel and run faster.

Tagged with: , , , ,

My timeline of text editors

I’ve been coding on and off since the beginning of the 90’s and along this journey I’ve been stumbling over quite a few editors and IDE:s. In this post I will go through some of them. It’s a selection of editors I either very much enjoyed working in or hated with a burning passion. I very much doubt that anyone might find this interesting or useful, but it’s sometimes nice to walk down memory lane.

Edlin (1980) (DOS)

edlin

Edlin.exe – Line editor, very archaic.

Edlin was the very first text file editor I used. Since it was shipped with MS-DOS 3.30 it was the only editor on the early PC’s DOS era. You could only edit one line at the time. It was all-right if you needed to tweak the autoexec.bat and config.sys, but to write anything complex and large would be unthinkable. If I wasn’t such a persisting child, edlin would probably had me losing my interests in computers.

What I like(d) about Edlin:

  • Nice with a bundled editor in DOS.

What I dislike(d):

  • Too bad the bundled editor was Edlin.
  • Unusable

What I use(d) it for:

  • Editting autoexec.bat and config.sys

More about Edlin on Wikipedia

MS Editor (1991) (DOS)

edit

MS Editor – Capable editor for its time.

MS Editor was a huge step up from Edlin with menus, search/replace, copy/paste functions and pretty easy to edit large files. What many people doesn’t know is that edit.com was actually the QBasic editor but stripped down. Edit became a proper standalone product with MS-DOS 6. I mainly used Edit for editing batch files in DOS. I would actually say that this editor was much better than Notepad we got with MS-Windows.

What I like(d) about MS Editor:

  • Copy, Paste, Search, Replace, multiple buffers

What I dislike(d):

  • Not much, extremely simple, no features for coding.

What I use(d) it for:

  • Editing batch files
  • Reading text files
  • Writing text files

Read more about the MS Editor on Wikipedia

QuickBASIC (1985) (DOS)

qb45

Quick Basic – The first IDE I used. You can see the resemblance to MS Editor.

In the beginning it was BASIC. As a kid this was the first programming language I came across. Me and my younger brother wrote an endless amount of small text games and some programs that painted some graphics on the screen. The difference from QBasic was that on Quick Basic you could compile your BASIC source code to executables and show them on a friends computer.

What I like(d) about Quick Basic:

  • Multiple buffers, Debugger

What I dislike(d):

  • Syntax highlightning

What I use(d) it for:

  • Writing programs and games in Basic

Read more about QuickBASIC on Wikipedia

Turbo C++ 3.0 (1991) (DOS)

tc_000

My first attempts in C++ were made in this IDE

 

My passion for writing text games didn’t seem to end. At this time I’d discovered MUD games. Which are the uninitiated a type of text multi-player games with monsters, quest and parties. Imagine it as a MMORPG in pure text. This made me start to write text games in Turbo C++. The result must have been the weirdest C++ code anyone ever seen, but I really enjoyed it. The IDE was very easy to understand and use.

What I like(d) about Turbo C++ 3.0:

  • Simple, multiple buffers, search / replace, project based
  • Easy to use with compilation and debugging

What I dislike(d):

  • Not much at all,

What I use(d) it for:

  • Learning C++
  • Writing programs

Notepad (1985) (Windows 1 and forward)

image2KU

Notepad and me was responsible for my first homepage.

Everyone has tried to write some kind of script code in this editor. I actually think I made my first webpage in notepad. But it really is a crappy editor. I don’t know how many times people has edited “nice formatted” XML data in notepad in UTF8, saved it and sent it back to me. The result is terrible, the format is scrambled, the UTF-8 characters are replaced or garbled. But to be fair, it’s not an editor, it’s a “notepad” and I guess it’s allright for taking notes…

What I like(d) about Notepad:

  • Quick to open and read text files.

What I dislike(d):

  • Bad default editor
  • Lack of features

What I use(d) it for:

  • Reading text files
  • Writing my first HTML documents

Read more about Notepad’s history on Wikipedia

Notepad2 (2004) (Windows)

Notepad2-4.0.22-Classic-C

Upgrading the notepad with notepad2

Notepad2 fixed everything that was wrong with the original notepad. I always used to replace the original notepad with this one on my Windows computer.

What I like(d) about Notepad2:

  • Multiple buffers
  • Very fast
  • Syntax high lightning
  • Support for many languages

What I dislike(d):

  • Sadly only for Windows

What I use(d) it for:

  • Same as notepad
  • Writing PHP scripts

Visit website
Read more about Notepad2 on Wikipedia

Macromedia Dreamweaver (1997) (Windows / MacOS)

DW8

Macromedia Dreamweaver – Great application, bad editor

I made a lot of web sites in Dreamweaver in the early 2000. At the time it was a great tool to quickly make a design that worked in the majority of browsers. It didn’t generate the nicest code to edit, but it worked.  I left Dreamweaver to learn CSS and HTML properly.

What I like(d) about Dreamweaver:

  • Easy to use
  • Generates a lot HTML

What I dislike(d):

  • Generates a lot HTML
  • Not really a good code editor

What I use(d) it for:

  • Creating web pages
  • Coding PHP
  • Editing HTML

Read more about Dreamweaver on Wikipedia

Midnight Commander Editor (1994 and onwards) (Linux / Unix)

mcedit

Midnight Commander Editor – great for editing conf files on servers.

I love mcedit, which comes with the Midnight Commander package. Midnight Commander itself, is a great file browser with support for ftp/ssh and lots of other great stuff. The editor supports syntax high lightning and is a pleasure editing small scripts with. Where a lot of people use nano or vi to edit their system files I use mcedit.

What I like(d) about MCEdit:

  • Syntax highlightning
  • The simpleness

What I dislike(d):

  • Single buffers only

What I use(d) it for:

  • Editing configuration files on servers
  • PHP/C++

Read more about Midnight Commander on Wikipedia

Geany (2005) (Linux / Windows / MacOSX)

vz7mq

Meet Geany, your new notepad replacement

Geany is my all time favourite lightweight editor. Nowadays I only use it for very simple things; taking notes, copy pasting code snippets and so on. It’s sad because Geany’s capabilities stretch a lot further than that. It supports a ton of different file formats, with syntax high lightning and simple auto completion. I’ve been using Geany on and off since 2006 and in the start I wrote a lot more code with Geany.

What I like(d) about Geany:

  • Lightweight
  • Very simple to use
  • Supports many fileformats
  • Built-in terminal support
  • Multiple buffers

What I dislike(d):

  • Not a thing

What I use(d) it for:

  • PHP programming
  • Viewing text documents
  • Taking notes

Visit Geany’s  Home Page
Read more about Geany on Wikipedia

Netbeans (1996) (Multiplatform)

nb_php

Working with PHP projects in Netbeans is a breeze

I love this IDE when coding PHP. It’s easy to setup and easy to tweak. The editor started out as a student project which later was commercialized and bought up by Sun MicroSystems (now Oracle).

What I like about the IDE is the great code completion. It also gives you hints about what is considered good code standard, which is a great help. I almost dare to say that this IDE made me a better developer when it comes to keeping your code looking clean and readable. It also has support for XDebug so you can profile and debug code from the IDE.

But the core strength of the IDE is coding in Java. It has a GUI builder and you can quickly create your own small desktop application.

The downside is that Netbeans is a fairly large application if you work with very small things and the lack of Python support.

What I like(d) about Netbeans:

  • Great with bigger projects in PHP
  • Comes with a GUI builder for Java
  • A lot easier to configure than Eclipse
  • Comes with a debugger and profiling tool.

What I dislike(d):

  • Supports a few picked languages
  • Maintained by Oracle

What I use(d) it for:

  • Bigger PHP projects
  • Java programming

Visit Netbeans’s web site
Read more about Netbeans on Wikipedia

Emacs (1976) (Unix, Windows)

emacs

For the hard-core programmer (Read: über unix geek)

Emacs is not an editor, it’s more of a life style. Some developers are rumoured live in Emacs. I first heard of Emacs when tinkering with Linux in the late 90’s, but never used it until now. Since Emacs’s learning curve is like a brick wall.

The strengths of Emacs, are all the different modes. One mode helps you code Python, another one C++ or another mode that is used as a file browser. The whole idea is to use Emacs as all-in-one environment, where everything can be tweaked and set up the way the user wants it. I have spent hours to find LISP code snippets that just do that thing I would like it to do and tweak it in my own way. There are a lot of help to find online. You can also extend the editor almost endlessly with packages.

I both like and dislike Emacs. Its full-screen mode focuses fully on the coding; without menus and other distractions. It’s ability to have many buffers also helps when you’re debugging or need to edit two/three files at the same time. On the downside is that I’ve never managed to get code completion working. Which usually is very handy if you sit on a large code base and trying to recall that weird name of a function.

What I like(d) about Emacs:

  • Fully customizable
  • Not many distractions from coding

What I dislike(d):

  • Need to know LISP in order to harness the good stuff
  • No good PHP support
  • No good code completion support

What I use(d) it for:

  • Coding in C/C++
  • Coding in Python
  • Coding assembler

Visit Emacs website
Read more about Emacs on Wikipedia

 

Tagged with: , , , , , , , , , , , , ,

Status update!

You haven’t noticed, but the blog hasn’t been updated for quite some time! The main reason is that on the 22nd of December 2014 I became a dad to two very early born children. Everything has gone well, but we’re very tired.

However I’m back at work and I’ll try to re-ignite some sparks into this blog when I find some topics of interests. But a lot of spare time “work in progress” will be put on a shelf.

Insanity + OS Dev = TempleOS

Sometimes you stumble upon interesting but yet very strange projects. TempleOS is absolutely one of those. It’s been developed by Terry A. Davis who has single handily been working on his Operating System project for over ten years. During these ten years it’s been called LoseThos and SparrowOS.

TempleOS is quite special since you are writing code directly towards the hardware and the whole point is to produce an OS that is more simple to understand. It has taken a lot of  inspiration from what the Commodore 64 meant for programmers in their teens, who are now in their 30’s and 40’s: A hackable OS for “recreational programming”.

But that is not really what makes TempleOS special is probably its creator. Terry is diagnosed with schizophrenia and there are times he doesn’t make any sense with his random angry outbursts, flavoured both with an unhealthy portion of profanity, racism and endless paranoia. Terry on twitter: https://twitter.com/TempleOS

He’s been actively banned from a myriad of forums due to the fact he spam irrelevant and offensive posts. This is a bit tragic since I figure Terry possesses oceans of knowledge to share, regarding OS development.

I found this thread which is a rare case where Terry gets some room and is being tolerated, and the participants in the thread doesn’t ridicule him. This gives him enough room to actually construct answers that makes somewhat of a sense. See: http://www.metafilter.com/119424/An-Operating-System-for-Songs-from-God

Even stranger

TempleOS is a very religiously Christian operating system. It’s filled with biblical passages and features, such as “Talk to god”, “Offerings to God” etc. In one of his videos, Terry talks about listening to God via TempleOS own random generator which is to be fair pretty strange!

The cause of the low resolution (640×480 with 16 colours), was apparently also a size that is easy enough to use and understand, according to what God has been telling Terry. He also wrote his own JIT compiler in a language he calls HolyC, which is a modified C/C++ language. This is used the same way under Temple OS as Basic was used on the Commodore 64, right from the start on the prompt.

It’s very hard not to be impressed what he’s actually made:

  • A 64-bit multitasking operating system
  • A compiler, AOT/JIT which is build in to his shell
  • A vast range of tutorials and games (including 3D games) for people that want to start tinkering with the OS

The idea of making an OS that is simple for the programmers is of course a cool idea. But I personally believe the people who still are tinkering with low-level programming are already hacking on Ardunio or RaspberryPI. And not to forget, the Commodore 64 demo-scene is still quite active and it’s super impressive to see how hackers still find new ways of pushing the 8-bit MOS 6510 processor and the 32 year old hardware further to do amazing stuff. (Probably a Commodore 64 post will come up later)

TempleOS is an interesting project. It’s certainly a mountain of work put into it, but who is it for? Probably mostly for the author. The Christian theme will put many persons off, since it’s like an OS that’s been designed by Salvadore Dalí in secret collaboration with Jehovah’s Witnesses. I like the idea of having an OS that’s built simple so programmers can learn more about the assembly and code behind, but I believe the audience and its users are elsewhere.

Read more about TempleOS:
http://www.templeos.org/Wb/Accts/TS/Wb2/TempleOS.html

Tagged with: , , ,

Internet Explorer 6, all over again

Chrexplorer 6In the period 2001-2005, Internet Explorer 6 was the most popular web browser around. Even the colourful plastic Apple Macs had their own Internet Explorer version as the default web browser.

Internet Explorer 6 was introduced with the new Microsoft Windows XP (which most of us had installed as the Devils 0wn “sneak preview”.)

The first web responsive* web applications were made around this time. And many applications were specifically written for IE6. This was mostly because the support of XMLHttpRequest (see AJAX). Since version 5, XMLHttpRequest had been a part of Internet Explorer and with version 6 the support had grown up to be fairly reliable. Other browsers had their different implementations of this technology too, but most people used and developed with Internet Explorer, so yes, the developers were lazy.

So as I see it, this one huge reason why Internet Explorer became so big. At the time, there were not really any competition. Netscape Communicator that was closest but it had just been up bought by AOL and been transformed to an ad possessed monster, with massive memory consumption issues. The success of IE6 was of course also helped that Microsoft shipped it as the default browser in Windows XP.

But happiness if brief

In the end of the last decade, Internet Explorer had been swiftly overtaken by new generations of web browsers like: Mozilla Firefox, Opera, Google Chrome and Safari to name a few. Internet Explorer had made many enemies mostly among developers and web designers. And for it’s users; Internet Explorer had become a personal ad-toolbar repository.

So how did this happen?
When developing web applications and web pages, Internet Explorer had to be treated as the “special-kid”. With it’s own exception code and CSS-hacks in order to work around its handicaps. While the other web browsers worked (most of the time) with the same CSS/Javascript and HTML.

The web designers struggled with Internet Explorer since it didn’t have support for transparent PNG-files until version 8 (2009)  and even that support was pretty bad. (Try change the opacity on a PNG image and you will see what I mean.)

For personal web project it was easy to disregard the support for the ancient Internet Explorer 6. But for web applications that met a larger audience,  you still had to support IE6. Most larger business and institutions were still jailed to Internet Explorer 6, since their old Intranet applications didn’t work with anything else.

Many web developing companies used  to add a separate clause in their contracts. That if the project required Internet Explorer 6 support, it would also cost 20% extra in development costs. (Tears and frustration included).

Today

Thankfully most of the web applications that were made in early 2000 has been upgraded. And the support for he older versions of Internet Explorer is not needed any more, even the newer versions of Internet Explorer plays along fine today.

There absolutely a myriad of things that is happening in the field of web development today. Especially on the front-end side.But the implementations in the different web browsers are starting to stray away from each other.

For instance there are many HTML5 and WebGL apps that only runs on Google Chrome. Since I’m using Linux most of my days I’m a bit pissed off that Adobe stopped supporting their Flash player for Linux. What we’re left with is a bug filled memory hogging plug-in, that most people would be wise to un-install. How ever Adobe has been working together with Chrome to implement Flash support in Linux, but only Chrome’s “Google version”, not the open sourced Chromium.

The latest news I read about Google and Adobe is that they’re planning to stream a Photoshop app via Chrome.
http://chrome.blogspot.dk/2014/09/adobe-joins-chromebook-party-starting.html

This is of course exciting, but in my eyes, Chrome starts to become a lot what Internet Explorer 6 was. “The preferred” browser and Exclusive functionality. I hope that every web developer and designer won’t repeat the same mistakes that was made 15 years ago and continue to support the majority of modern web browsers. The web is and should be a multi platform, accessible for as many as possible.

Personally I wish people would grow more aware about Google. It’s not the creative, innovative, loving start-up company we used to know.

  • It’s a billion dollar business.
  • Its business is you and what you do on-line.
  • And Google Chrome is just one of its tools.
* Receiving, displaying and updating data without to reload the web page.
Tagged with: , , , , ,

Exposing my own php object generator – PowerDB

surveys-complex
Once upon a time I wrote my own database ORM. It is heavily influenced by the PHP Object Generator. Which was a very easy to use tool for generating database objects in PHP.

Sadly POG had its flaws:

  • Low reliability under pressure, a lot of weird things exhausting the db connections
  • Lousy code (wildlife PHP code)
  • Only produced direct queries without security IE. INSERT into books VALUES (‘Title’, ‘2023’, ’24’);
  • Only MySQL support
  • No development

So the main reason why I wrote PowerDB was because that I needed support for both PostGreSQL and MySQL for different setups. I was also tired of trying to hack the miserable PHP code that POG written in. It resulted in a PHP library that can do the following:

  • Creating database relations such as CHILD, PARENT and SIBLING
  • All queries are made through PDO statements to prevent injection and other errors.
  • Fast (if used in the right way)
  • Designed to produce maintainable code
  • Support for transactions via PDO

In PowerDB you generate base classes via a web interface. The web interface also contains a package where you can try out your models and generate the database tables. It actually enables you to produce some complex data models. I wrote a project using 40 tables.

The whole idea is that PowerDB generates base classes which are meant to be derived from, so you can put your custom code in the derived classes. To make it easier to re-generate models that needs an extra field in the database.

Here is a brief coding example how it supposed to be used:

The code is pretty self explanatory. PowerDB gives a clean way of working with database entities. The Author object has child objects books. Each book has a parent called Author, which is set automatically. The library object has authors as child objects. Since all the objects are related to the Library object I only need to call save on the “master” object and it will create an author, two books and a library.

It also supports sibling relationship which actually means that an entity (for instance a book) can point to many different entities. For instance a book can point to related books.

To query the data later on is fairly simple:

The code is just an idea on how the developer will use the different objects to access data. (Of course you should include checking when writing proper code.)

I’ve been using PowerDB in a lot of small projects, and I believe it’s a great tool for small to medium PHP projects. There are some things that would be nice to have in the library as support for composer distribution, namespaces etc, support for more databases.

If you find this interesting, checkout the PowerDB generator or browse/fork the code for the generator on Bitbucket. Maybe in the future I will get around to write a simple manual for it.

Tagged with: ,
Top