Text editors and some shell scripting
In the last sections we learned how to use commands that move and work with files, but we haven't learned how to actually edit files, which is what we'll be learning in this section. Text editors are used for editing files and there are many text editors to choose from. You've probably already used some text editors, like Notepad or TextEdit, as these are the default text editors on a Windows or Mac OS. In this section, we'll be taking a look at a lot of different text editors. It's good to idea to be familiar with a lot of text editors because if you're working on a server that doesn't have the text editor you're used to, it's good to have another text editor that you're somewhat familiar with.
The first text editor we'll be looking at is the 'vim' editor, as it's used ubiquitously across different OS's and servers. Next, we'll briefly introduce some other text editors; Nedit, Gedit, Pico and Emacs. Becoming proficient in all of these text editors is a bit excessive, and it's not a criteria for this course. It is, however, a good idea just to know that many text editors exist. If you're ever in need of using one of these text editors, it's pretty straightforward to 'google' your way to a guide, or use some of them links presented in this section.
In this section, we'll take a sneak peak at shell scripting. At the simplest level, shell scripts are multiple commands saved within a file. When the file is executed, the commands within are run. This is useful when you need to do the same workflow on multiple tasks, which can be quite labor intensive. On a more complex level, shell scripts can be equipped with syntax like for and while loops, and you can actually do programming in shell scripts. However, doing large programs in shell scripts isn't recommended as BASH and other shell syntaxes can be quite difficult to learn and read. Instead, you'd be better off using a programming language such as 'Python', 'C++', 'Java' etc..
Also, when making shell scripts it's a good idea to keep reproducibility in mind. This can be done by giving your script a proper title, that adequately explains the function of your script. It's also paramount to make comments in your script, which makes it easier for others (and yourself in half a year) to understand and use your script. It also makes it easier to change the script. In the next section, we'll go into detail with comments and the other basics of shell scripts.
In the last bit of the section, we present a guide on installing 'Sublime 3', a user-friendly and a nice text editor for programming. We'll set it up so it can be run from the command line on your Windows or Mac computer.
Underneath are some of the commands we'll be using in this section.
| Unix Command | Acronym translation | Description |
|---|---|---|
| alias <alias_name>=<The stuff you want to make an alias for> | - | Creates an alias called alias_name for what you've inserted on the right side of '=' |
| source <FILE> | - | Executes the contents of a file in current shell. Changes made when the file is run will be permanent until changed. It is synonymous with Prompt$ . <FILE>. |
| bash <FILE> | Bourne again shell | Bash will execute <FILE> as a different process. This way, changes that occur while the file is being executed cannot affect your shell. |
The command, alias, is useful when you want to simplify long commands or if you often need to go to faraway directory with a long file path. You can for instance, make a directory that allows you to go to your desktop quickly.
Prompt$ alias desktop='cd filepath_to_Desktop/Desktop'
This will create an alias called 'desktop', which will change your directory to your Desktop. These changes, however, are only temporary and the next time you open your terminal it won't work. In order to make it a permanent alias we have to edit in what's called the .bashrc file, which is located in your home directory,
Prompt$ cd ~ Prompt$ ls -a <home directory>
It's a hidden file, indicated by the dot symbol '.', which is why you have to use the command option -a to see it. On the MAC OS, this file is called .bash_profile instead. The .bashrc file contains the commands that are run when you start a bash shell, and the 'rc' in .bashrc is actually short for 'run commands'. Therefore, to make the alias permanent you need simply append the alias to .bashrc.
Prompt$ echo 'alias desktop='cd filepath_to_Desktop/Desktop/' >> .bashrc
This will ensure that the next time you open a terminal, the alias command will be run.
Instead of closing and opening your terminal to restart it you can instead type,
Prompt$ source .bashrc
which will ensure that the commands within .bashrc are run. Alternatively one could write '. .bashrc', which is synonymous to 'source .bashrc'
The bash command, which is the command we'll be using to execute shell scripts, is somewhat similar to source because it also executes files. But the difference is that bash executes files in a separate process than the terminal. Remember, the terminal is a program itself and an ongoing process. This way commands that might change the settings of the terminal are only set during the execution of the file. Therefore you would never write,
Prompt$ bash .bashrc
when restarting your bash. In figure 3.1 we show the difference between source and bash. The file, test.sh, contains a variable called Variable1. Variables can be assigned with the syntax, variable_name=variable_value, and $ is a special character that let's the shell know, that subsequent string is a variable. If the $ wasn't used, echo would output 'Variable1'. In the figure we see that when bash is used to execute test.sh, the variable is not saved. Conversely, when source is used, the variable is saved.
Vim editor
The vim editor is one of the most widely available text editors. It's by default installed on most systems and therefore worth knowing. Start by following this link to a couple of introductory videos on the Vim/vi editor Vim/vi editor tutorial videos.
To open a file with the vim editor simply type,
Prompt$ vim <FILE>
An important vim editor feature, is that there are multiple mode from which different commands can be issued. The 3 most important modes; are command mode (also called normal mode), insert mode and visual mode. After having opened a file with vim, you'll start out in the command mode. You can move your cursor around with the arrow keys. There are other more elaborate ways for navigating, such as moving an entire line forth/back, but unless you're constantly using vim this sort of navigation can be difficult to remember. From the command mode you can issue commands, like entering insert and visual mode. Before we do that, however, it's important to know how to exit the vim editor. While in the command mode, type
:wq
which will appear at the bottom of vim editor display. This will save your file and exit. If you simply want to exit, type
:q
which will exit your file without saving. If this doesn't work, it's probably because you're not in command mode. You can always return to command mode by typing the 'Esc' key.
While in command mode you can type 'i' and 'v' to enter insert and visual mode respectively. Within the insert mode, you can write and delete text as you would in any text editor. The visual mode is used for highlighting text, that you want deleted, copied and replaced. From the command line you can actually enter 3 types of visual modes; visual ('v'), visual line ('shift-v') and visual block ('ctrl-v'). These difference visual modes are simply different ways for which text can be highlighted. First set your cursor to where you want to start highlighting, then enter one of the 3 visual modes. You can then choose the text that you want highlighted by moving the cursor. It is within the visual mode, that you can copy/cut/paste content of the file. To do this, first highlight the text you want cut/copied by entering one of the 3 visual modes. From here, you can then press 'd' to cut and 'y' to copy. Subsequently, you can move to where you want the content pasted with the arrow keys, and press 'P' to paste before the cursor, and 'p' to paste after the cursor.
If you've done a something that you shouldn't have done within vim, you can undo your previous action by typing 'u' while in command mode. Also, you can move your cursor to a specific line by typing <Line number>-G while in command mode. This will move your cursor to the line specified by line number. The number of utilities for the vim editor is long and it's likely that you might never have use for any of them. For now, what's important is that your able to open a file, edit the file in insert mode, copy/paste content with visual mode and save/exit the file.
Exercise 1: Making a simple shell script with vim.
Datafile 1: Pseudomonas Aeruginosa 16S rRNA Genebank file
Datafile 2: ex1.acc
Datafile 3: ex1.dat
1. Download the 3 datafiles.
2. Create a file with the ending '.sh', which makes it a shell executable file, in your current directory.
3. Open it with vim and go into insert mode.
4. While in insert mode type in a command that will create a sub-directory to your current directory.
5. On the second line, type in a command that copies the 3 data files to this subdirectory.
6. On the third line, type in a command that merges the 3 data files and saves it as merged.dat.
7. On the fifth line, type in a command that will delete the subdirectory that you created.
8. Exit the file and execute the file from the command line by typing,
Prompt$ bash <file.sh>
.vimrc file
The .vimrc, where the 'rc' is short for "run commands", is a file that contains commands that are run whenever you open a file with vim. It's located as a hidden file within your home directory or if you on a MAC OS, it should be located at /usr/share/vim/vimrc. You can think of it as a settings file for the vim editor. By default your .vimrc file should be empty and customizing your vim editor is a task completely up to you. Customizing .vimrc is simple, just open it with vim as you would any other file. Underneath is link to a video, that explains some basic settings you could do in your .vimrc file. Tutorial on some basic .vimrc settings
You can actually also download a premade .vimrc file that gives your vim editor a lot of extra utility. However, as you might someday be working on computers where these utilities aren't available, it's a good idea that you first become efficient with the default vim editor.
Other text editors; Nedit, Gedit, Pico and Emacs
Here, we present some 'very short' introduction of some other text editors that you might run into.
- Nedit
Nedit, short for 'Nirvana editor', has an interface similar to that of text editors found natively on Windows and Mac computers. It has some functionalities that it excels at, which are listed have been listed and explained in this link Nedit.
- Gedit
Gedit is the default text editor in GNOME desktop environments. We haven't talked about 'GNU project' and 'GNOME' yet so this might be a good time. 'GNU' is not an acronym, but another name for a wildebeest (remember those things that killed the lion king, yep), which was like the mascot for the project. Essentially, the 'GNU project' was a mass-collaboration project that started in September 27 1983 with the goal of giving computer users more control of their computers by providing free software to users. GNOME, short for 'GNU Network Object Model Environment' was one of the results of this project, and it's actually the default desktop environment used by many Linux distributions, for example, Ubuntu uses it. GNOME has provided a help guide, that can help you get started if you want to learn gedit- Help guide to gedit
- Pico
Pico, short for 'Pine composer', is text editor for UNIX based computer based systems. The good thing about learning Pico, is that it's simple and easy to learn. The downside to pico is that it doesn't have as many features as for example the 'vim editor' which we'll be learning later. Also, pico has a clone text editor called 'nano' which was created as part of the GNU project due to unclear redistribution terms. So if you've learned 'pico' then you've also learned nano. If you're interested in learning pico, a good start would be to watch this video Pico text editor introduction.
- Emacs
Emacs was initially released in 1976, and has since undergone many tweeks and updates. Among programmers, there exists a something called the 'Editor war', which is like a rivalry between 'emacs' and 'vim' Editor war. It's, however, considerably more difficult to learn 'emacs' compared to 'vim'. To get you started with emacs, you should have no trouble finding various tutorials on youtube.
To install emacs, it's best to use UNIX commands sudo and apt to ensure that it's installed properly. Here's what you would type in your terminal,
Prompt$ sudo apt -y install emacs
Sublime 3
Windows
Sublime is a pretty cool text editor with some nice features, especially for programming purposes. There are two good ways of setting up Sublime 3 in a Unix terminal on a Windows computer. One way is to use a program called Xming coupled with the use of the Unix commands apt and sudo. But as we haven't covered these commands yet (they will be presented in the section Advanced Packaging tools), we won't be using this method.
For the other method, start off by downloading the windows version of Sublime text by following this link Sublime download link. When installing Sublime 3, make sure you know where you're saving it (by default, windows will save it in \Program Files). Once you've successfully installed Sublime 3 start your Ubuntu WSL terminal. From your terminal, you can directly access sublime text by specifying the filepath to 'subl.exe',
Prompt$ /mnt/c/Program\ Files/Sublime\ Text\ 3/subl.exe <FILE>
will open <FILE> with the sublime text editor. The '\ ' are how spaces are written in a command line. This can be confusing so spaces are often avoided in filenames by replacing spaces with '_'. We can avoid having to write such a long filepath everytime we use sublime by using the command alias,
Prompt$ alias subl='/mnt/c/Program\ Files/Sublime\ Text\ 3/subl.exe'
which creates an alias for what's specified after '='. This way, typing subl<file>, will open the file in sublime text editor. This alias, however, is not permanent and the next time you start your terminal you'll need to write it again. In order to make it a permanent alias we have to edit in what's called the .bashrc file, which is located in your home directory,
Prompt$ cd ~ Prompt$ echo 'alias subl='/mnt/c/Program\ Files/Sublime\ Text\ 3/subl.exe >> .bashrc
Now everytime you start bash the alias command will be run. In figure 3.1, we show the commands that need to be executed in the terminal.
Mac
For Mac computers, the installation method for Sublime 3 is similar to the Windows installation. You can download the mac version of Sublime 3 by following the link, Sublime download link.