Wednesday, 27 January 2021

Learning the basics of Command Prompt

This is a tutorial I found that I included in a lecture I did on the 8th of March, 2017 at the Department of Civil Engineering and Geomatics at Cyprus Unviersity of Technology. The idea was to introduce the students to the basics commands of Command Prompt before explaining to the how to use my open source software DASOS (http://miltomiltiadou.blogspot.com/2015/03/las13vis.html). By quickly reading thought it, I think it worth sharing on my blog. :) 

 

Introduction

This tutorial gives you an overview of the basic line commands that can be executed using the  Microsoft Windows Command Prompt. By following the procedures of this tutorial, you will learn how to:
1. Open Command Prompt on Windows
2. View the contents of a directory
3. Change directory
4. More useful commands
5. Executing multiple commands using .bat files


1.Opening the Command Prompt window

The Command Prompt is the command-line interpreter on Windows machines. For example, it allows you to execute line commands for opening or modifying files. For windows 7 and  lower, the Command Prompt is found by searching ”Command Prompt” on the Star Menu  Search bar (Figure 1a). For windows 10, you will find it by pressing right click on the start  icon (Figure 1b). Inside the black window that appears, we can write and execute commands. Figure 2 shows a Command Prompt window. The last line shows the path of the working  directory, which in this case is C:\Users\Milto. The follow sections give an overview of a few basic commands.





2. Viewing the contents of a directory - <dir>

In order to view the content of the working directory we use the command dir. The name of the command dir is derived from the word ”directory”. Type the following at the command prompt and press ”ENTER” from the keyboard (Please
note this in this tutorial the $: shows the start of a command and it should be ignored.):

$: dir

Once the ”ENTER” is pressed, a list similar to the following appears:


This is the list of all the files and subfiles of the working directory. The <DIR> label indicates that the listed item is a directory itself. If the working directory is not a drive (e.g. C:\), the first two directories ("." and "..") are always listed. The "." directory is the current working directory (C:\Users\Milto) and the ".." is the directory of the folder that contains the working directory (C:\Users). When a directory contains many items, the tag /p is very useful. For example, type the
following command: 

$: dir /p 

This will print a page of the directory list and the next page appears once ”ENTER” is pressed. 


3 Changing directory - <cd>

Moving from one directory to another is essential and the command cd (named after ”change directory”) is responsible for that. From the working directory (C:\Users\Milto), you can move to the subdirectory Documents by typing the following: 

$: cd Documents 

Or you may include the entire path of a directory. For example:  

$: cd C:\Users\Milto\Documents 

By the way, when typing in Command Prompt you may use the tab button to quickly fill the name’s of directories and files. Try typing: 

$: cd C:\Us 

and then press tab. It will automatically be filled to cd C:\Users. If more than one options apply, we can looping through them by pressing the button tab multiple times. Additionally, it is essential to be able to move from a working directory backwards; to the folder containing that working directory. This is done by the following command: 

$: cd .. 

As mentioned before, the directory ".." is the directory of the folder that contains the working directory. Therefore, by using the command cd following with ".." , we can move backwards one folder. Similarly, the following command does nothing because it brings you to the current directory, which is the "." : 

$: cd . 

The final command related to changing directory for this tutorial is the following:

$: cd \ 

Type the above command in Command Prompt. This command brings you to the root directory, which should by C:\. Therefore the Command Prompt should now show: 

C:\> 

Please note that for the final command a baskshlash (\) is used and not a forward slash (/); 


4.Quick Overview of some useful commands

There are numerous commands that can be executed from Command Prompt. Here a number of them are listed. 

To create a folder, use the command md (make directory) as follow. Please replace the <folderName> with the name of the new folder to be created. 

$: md <folderName> 

With the command rd (remove directory), an empty folder can be deleted: 

$: rd <folderName> 

For non-empty folders the tag \s should be added at the end as follow: 

$: rd <folderName> \s 

For renaming a file: 

$: ren <oldName> <newName> 

For deleting a file: 

$: del <filename> 

For copying a file: 

$: copy <file> <destination> 

An example of this command is the following which copies the file.txt from the directory C:Users into the C:\HelloWords: 

$: copy c:\Users\file.txt c:\HelloWorld 

Similarly the structure of the command that moves files is the following: 

$: move <file> <destination> 

The parameter NUL represents emptiness/null. Here, there is a hack for quickly generating an empty text file using the copy command and NUL: 

$: copy NUL emptyFile.txt 


5.Executing multiple commands using batch (.bat) files

A batch file is a script file in Microsoft Windows. It consists of a series of commands, which are stored in a plain text file (most commonly with extension .bat). These commands can be executed by the command-line interpreter (Command Prompt). Figure 3 shows an example of a batch file that uses commands explained in this tutorial. Each line that starts with :: is a comment and it is ignored at execution time. Comments also do not influence the interpretation

of the commands.

As mentioned before, batch files are plain text files and they can therefore been edited using a text editor. A good option for editing batch files is the gedit application, which colours the commands. If gedit is not available, then WordPad will also work fine, but preferably avoid using Notepad because usually it does not identify new lines and there is no indentation. To run a batch script file, just double click on the file and all the commands will be executed. Once the execution is done, the command prompt closes automatically. To avoid that you may include the pause command at the end of the file.


Tuesday, 3 November 2020

Co-registration of GeoTiff images with Full-waveform LiDAR data

In Remote Sensing, combining multiple sensors could be beneficial due to the increased information that could be used to train a classifier. Here, there is a tutorial with some scripts accosiated to help you co-register LiDAR metrics exported from the open source software DASOS [1] with GeoTIFF sattelitte imagery:

Link to open source software DASOS: https://github.com/Art-n-MathS/DASOS 



The “FOREST” project, with project protocol number “OPPORTUNITY/0916/MSCA/0005”, is co–financed by the European Regional Development Fund and the Republic of Cyprus through the Cyprus Research & Innovation Foundation.


Work Cited:
Miltiadou, M., Grant, M. G., Campbell, N. D., Warren, M., Clewley, D., & Hadjimitsis, D. G. (2019, June). Open source software DASOS: Efficient accumulation, analysis, and visualisation of full-waveform lidar. In Seventh International Conference on Remote Sensing and Geoinformation of the Environment (RSCy2019) (Vol. 11174, p. 111741M). International Society for Optics and Photonics.


Monday, 19 October 2020

Python script that creates a copy of the same folders structure

Hello, it has been a while since I last posted something. Today, I wrote a very short script but something that could be useful so I thought it worth sharing and keeping into my blog. So have you ever wanted to copy and paste multiple folders without their content! This is what the following python function with automatically do for you:
import os

def creatFolders(i_inDir,i_outDir):
   cdir=os.getcwd()
   os.chdir(i_inDir)
   dirs=[d[0] for d in os.walk(".")]
   os.chdir(cdir+"/"+i_outDir)
   for d in dirs:
      if not os.path.exists(d):
         os.makedirs(d)
   os.chdir(cdir)
The following scripts returns all the files from a direction with a given extension:
import os

def getFiles(i_inDir,i_ext):
    cdir=os.getcwd()
    os.chdir(i_inDir)
    images=[f for f in os.listdir(".")if f.endswith(i_ext)] 
    print (images)
    print ("Images in Dir")
    print (i_inDir)
    os.chdir(cdir)
    return images
This script was developed as part of the "ASTARTE"(EXCELLENCE/0918/0341) project, which is co-financed by the European Regional Development Fund and the Republic of Cyprus through the Research Innovation Foundation. | Powered By Copywriter WordPress Theme.

Thursday, 9 May 2019

Installation guide for Anaconda, Python, OpenCV and fmask for interpreting Sentinel Images



Download and Install Anaconda available at: https://www.continuum.io/downloads During installation, make sure you include Python in the path:
- Run Anaconda Prompt as an Administrator to be able to install new libraries and run the following commands:
:$ conda install -c conda-forge gdal

Install Jypiter Notebook:
:$ conda install -c conda-forge jupyterlab
Run JypiterLab:
:$ jupyter-lab

Install opencv:
:$ conda install -c menpo opencv
You may need to run the above command using an Administrator Command Prompt


For reading .nc images of Sentinel 3:
:$ conda install -c conda-forge netcdf4


Install fmask from Administrator Command Prompt as follow:
:$ conda config --add channels conda-forge
:$ conda install -c conda-forge python-fmask
or (to be tested)
:$ conda create -n myenv python-fmask
:$ activate myenv


Other dependencies:
:$ conda install -c anaconda scipy
:$ conda install -c anaconda numpy
:$ conda install -c conda-forge matplotlib
:$ conda install scikit-learn


For Doxygen Documentation please install it from here: http://www.stack.nl/~dimitri/doxygen/download.html


Every time you run your scripts, you need to run Anaconda Prompt as an Administrator and run "activate myenv".


Once you activate myenv, check they version of python that you are using. There is a bug in recent anaconda versions and when myenv is activated, your python is automatically updated to 3.6. You may use the following command to downgrade it:
:$ conda install python=2.7.8
Please note that this may be risky, since other applications may depend on Python 3.6.



Acknowledgement

This is part of the H2020 Research Innovation and Staff Exchange project SEO-DWARF with reg. no MSCA-RISE-691071. Website: seo-dwarf.eu

Monday, 8 April 2019

Histograms with R



This is an example of a histogram creation using R that I would like to save in my blog for quickly referring to it when required.


# Run the script using the following command:
# Rscript Histogram.r

# Define an array
arrayA<-c(17.6,16.8,33.6,28,33.6,28,40.8,37.6,38.4,30.4,25.6,23.2,28,16,15.2,24,16.8,32,28,15.2,15.2,28,28.8,15.2,24.8,15.2,14.4,29.6,38.4,19.2,27.2,37.6,15.2,33.6,33.6,28,18.4,17.6,26.4,26.4,36.8,24.8,32,19.2,16,33.6,32,16,16,30.4,16,37.6,16,25.6,27.2,28,24,26.4,26.4,20.8,16.8,26.4,28,32.8,24,15.2,15.2,16,27.2,12,40.8,38.4,40.8,40,15.2,37.6,17.6,17.6,27.2,14.4,15.2,20,19.2,26.4,27.2,14.4,31.2,27.2,28.8,15.2,15.2,14.4,14.4,28.8,24.8,14.4,15.2,14.4,19.2,31.2,18.4,28.8,17.6,17.6,17.6,17.6,17.6,17.6,32,31.2,32,46.4,37.6,40.8,39.2,17.6,17.6,17.6,18.4,17.6,17.6,34.4,34.4,16.8,16.8,15.2,39.2,40.8,29.6,42.4,40.8,40,38.4,42.4,15.2,15.2,16,15.2,34.4,14.4,14.4,14.4,30.4,42.4,48.8,14.4,32,28,28,14.4,14.4,25.6,22.4,29.6,28,31.2,26.4,26.4,25.6,14.4,14.4,14.4,18.4,19.2,19.2,18.4,39.2,15.2,30.4,28.8,33.6,32.8,15.2,33.6,32,32,32.8,31.2,33.6,15.2,24.8,40.8,40.8,39.2,26.4,25.6,18.4,18.4,40.8,37.6,19.2,19.2,37.6,19.2,28.8,28.8,24.8,28,15.2,14.4,31.2,19.2,18.4,19.2,19.2,19.2,32,37.6,14.4,12.8,30.4,15.2,14.4,40,27.2,30.4,38.4,20.8,40,20,20.8,40,41.6,32.8,20.8,20.8,39.2,20.8,20,36,20,19.2,34.4,32,20,20,30.4,26.4,21.6,32,15.2,15.2,28.8,24.8,29.6,17.6,27.2,30.4,33.6,13.6,33.6,35.2,27.2,28,16,15.2,15.2,15.2,28,31.2,38.4,25.6,38.4,29.6,15.2,15.2,32.8,33.6,33.6,26.4,14.4,28.8,34.4,33.6,15.2,32,40,33.6,13.6,15.2,37.6,35.2,14.4,32,15.2,33.6,18.4,22.4,38.4,18.4,36.8,14.4,25.6,14.4,33.6,15.2,26.4,24.8,28,36,39.2,14.4,33.6,11.2,15.2,49.6,35.2,36,46.4,46.4,14.4,46.4,12.8,45.6,15.2,41.6,14.4,41.6,14.4,13.6,37.6,12.8,39.2,41.6,14.4,12.8,13.6,14.4,14.4,14.4,18.4,14.4,32)


# 1. Open jpeg file

jpeg("/home/username/Documents/histArrayA.jpg")


# 2. Create the plot inside the file

hist(arrayA,breaks=seq(0,70,l=70))


# 3. Close and save file

dev.off()



The output of the above script is the following:



Acknowledgments:

The script of this post was written as part of the "FOREST" project with reg. no OPPORTUNITY/0916/0005. The "FOREST" project  is co-financed by the European Regional Development Fund and the Republic of Cyprus through the Research Promotion Foundation.

Friday, 8 February 2019

Reviewers please be kind!

Academia is a very competitive world and it is something that many people, including my self, do not understand when they start a doctorate degree. Competitiveness is OK since we learn how to accept failure and be persistent in getting this work publish. But sometimes I feel disappointed when I receive the comments of the reviewers, not because they rejected my paper, but because they did not even read the entire article and it is clear from their comments! I think more people may resemble to this experience. So, I decided to write this post and ask reviewers to carefully read the entire articles, be kind and encouraging while recommending ways of making the work publishable.

Personally, I am new researcher and I have not written enough papers to fully understand the process and avoid small mistakes. But if reviewers do not help me improve myself then I will never be able to progress my career in academia.

Here are some comments that I found disturbing:
-  This is from a paper that have been rejected before even been reviewed: "a link to bird diversity is offered as motivation, the link to remote sensing is not pursued further in the manuscript". The article was about proposing a new methodology for detecting dead trees from full-waveform LiDAR data. If LiDAR is not about remote sensing then what could it be? By only rephrasing the abstract and submitting it to another high impact journal the article was published with minor corrections, indicating that the editor had not read the article.

- "The LiDAR-specific complications of mapping full-waveform data to scalar volumes are entirely handle by DASOS. It means means that the manuscript oversells its contribution" The reviewer missed the part that DASOS was implemented by the authors of the manuscript to make the research possible!

- "Evaluation of the method is completely insufficient," could have been phrased in a kinder way, especially when the reviewer request comparison with Canny Edge that it included in the article. On top of that the reviewer states "While Canny was considered, its awful results in table 3 indicates that it was implemented incorrectly or using a poor choice of settings." How is it possible for the standard python opencv function for the Canny Edge algorithm to be incorrectly implemented? The reviewer could have request instead an explanation for the bad results, which exists: the Canny Edge includes a smoothing step and the gradient differences are low. Therefore, Canny Edge fails to detect many edges!

- The same reviewer questions the approach used to find k for k-means and that it may not be reliable, but missed the part that the mean shift was also implemented in the article that does not require the number of clusters (k) to be pre-defined. 


Do not get me wrong, there are reviewers that provide constructive feedback that help improve a paper. I just feel that more reviewers should act like that. Behind every article, there is a researcher who spent months reading articles, conducting experiments, stressing with every unexpected results, missing social events hoping that the code will work this time and spending nights overworking to finish with the writing. So be kind to them! I understand that I and every young researcher makes mistakes. My articles usually lack of presentation, but if reviewers are judgmental and badly criticizing my work without even reading the entire articles, then how will I improve myself? 






Wednesday, 21 March 2018

Course: Overview of LiDAR; system variations, data interpretation & applications



-        Gain an in-depth understanding of LiDAR concepts, systems and algorithms. 



Light Detection And Ranging (LiDAR)

Quickly progress to understand the state-of-art LiDAR research and systems development.
Become an expert on LiDAR data and systems with this step-to-step course, starting from scratch.
This video course gives detailed and broad information about how LiDAR systems works, their usability and interpretation.

According to Wanger et al, LiDAR is a growing technology used in environmental research to collect information about the Earth, such as vegetation and tree species. Earth observation images, acquired from satellites, have been used for years in earth monitoring. In respect to forest monitoring, satellite imagery does not contain information about tree height, diameter at breast height and stem density amongst many other important parameters for monitoring forest health at tree level. In the last couple of decades, LiDAR data acquired from airborne platforms has been increasingly used for forest monitoring, urban planning, archaeology, biodiversity and automated driving. Using this technology, the commercial forestry sector managed a 40% reduction of the expensive fieldwork that cost them millions of dollars annually.

Just listen to these videos and you will become an expert on LiDAR systems in a fraction of time!
Enhance your knowledge of Earth Observation and Remote Sensing along the way!
Check out the curriculum for the detailed contents of this video course!

Curriculum
Chapter 1: Overview of the course (2:56)
Lesson 1:  Overview of the course



Chapter 2: Introduction to LiDAR systems (10:30)
Lesson 2: How LiDAR systems work
Lesson 3: Introduction to discrete and full-waveform LiDAR
Lesson 4: Types according to the way they are carried
Lesson 5: Types according to the way the pulses are emitted

Chapter 3:  Interpretation of LiDAR data (6:50)
Lesson 6: Introduction to your first metrics (Digital Elevation Model, Digital Terrain Model and Canopy Height Model)
Lesson 7: Tree delineation using the Watershed Algorithm
Lesson 8: Further tree delineation approaches

Chapter 4: Full-waveform LiDAR data (11:35)
Lesson 9: Discrete versus full-waveform LiDAR data
Lesson 10: Comparison of data collected using the Leica ALS50_v2 sensor
Lesson 11: Echo Decomposition for peak point extraction
Lesson 12: Voxelisation of full-waveform LiDAR data

Chapter 5: LiDAR file formats (12:13)
Lesson 13: Introduction to binary files
Lesson 14: Discrete LiDAR LAS files formats
Lesson 15: Full-waveform LiDAR LAS file formats
Lesson 16: How to calculate the positions of the waveform samples
Lesson 17: The Pulsewaves file format 

Chapter 6: Sample of available software for interpreting LiDAR data (6:51)
Lesson 18: Sample of available software for interpreting LiDAR data

Chapter 7: Applications of LiDAR data (9:28)
Lesson 19: Biodiversity
Lesson 20: Forest health monitoring
Lesson 21: Urban planning
Lesson 22: Wood trade
Lesson 23: Archaeology
Lesson 24: Automated Driving

Chapter 8: Other Types of LiDAR Systems (2:42)
Lesson 25: Multi-Spectral LiDAR
Lesson 25: Atmospheric LiDAR
Lesson 26: Bathymetric LiDAR

What will you learn?
- Learn how LiDAR systems work
- Gain an in-depth knowledge of various LiDAR systems
- Understand the differences between discrete and full-waveform LiDAR data
- Acquire an understanding of many algorithms used for interpreting LiDAR data
- Learn how to tackle issues using LiDAR in various application areas
- Become aware of various available software able to process LiDAR data

Any prerequisites?
- Basic knowledge of Earth Observation (optional)
- A simple laptop or desktop computer to watch the lectures

Student Profile?
- Undergraduate & Postgraduate students
- PhD/EngD candidates
- Professionals
- Researchers and Academics
- Geospatial Analysts
- Remote Sensing Scientists