Python: Fast Fourier Transform (FFT) for speech processing using Scipy

Scipy provides the FFT and inverse FFT functions useful for sound processing.

Example:

import soundfile as sf

from scipy.fftpack import fft, ifft

with open('1.wav', 'rb') as f_wav:

    x, rate = sf.read(f_wav)

   

X = fft(x)

#The ifft function returns a complex variable, but the imaginary part is quite small to be neglected.

y_complex = ifft(X)

y = y_complex.real

sf.write('2.wav', y, 16000)

Reference:

Matlab: Discrete Cosine Transform (DCT) for speech processing (StudyEECC)

Python: Discrete Cosine Transform (DCT) for speech processing (StudyRaspberryPi)

Fourier Transforms (scipy.fft)

scipy and numpy inverse fft returns complex numbers not floats, can't save as wav (StackOverflow)

Python - How to use Soundfile to read and write WAV and FLAC files (Study Raspberry Pi)

Python: Discrete Cosine Transform (DCT) for speech processing using Scipy

The DCT and inverse DCT may be used to convert a speech signal into the transform domain using real values and back to the speech waveform.

Example using Python and Scipy:

import soundfile as sf

from scipy.fftpack import dct, idct

#DCT and IDCT functions have to be normalized.

def dct_norm (block):

    return dct(block.T, norm = 'ortho').T

def idct_norm (block):

    return idct(block.T, norm = 'ortho').T

  

with open('1.wav', 'rb') as f_wav:

    x, rate = sf.read(f_wav)

X = dct_norm(x)

y = idct_norm(X)

sf.write('2.wav', y, 16000)

Note that the DCT and IDCT functions have to be normalized to obtain identical results to MATLAB.

Reference:

Matlab: Discrete Cosine Transform (DCT) for speech processing (StudyEECC)

scipy.fftpack.dct

Discrete Cosine Transforms (Scipy Tutorial)

In scipy why doesn't idct(dct(a)) equal to a? (StackOverflow)

Python - How to use Soundfile to read and write WAV and FLAC files (Study Raspberry Pi)

Python - How to use Soundfile to read and write WAV and FLAC files

WAV and FLAC are two commonly used sound file formats:

WAV - Waveform Audio File Format (Wikipedia)

FLAC - Free Lossless Audio Codec (Wikipedia)

The major difference between the two formats is that WAV is without compression and FLAC is for lossess compression.

In Python, to read and write WAV and FLAC files including export in a different format, simply use the code with Soundfile as below:

import soundfile as sf

with open('wav/1.wav', 'rb') as f_wav:

    x_wav, rate = sf.read(f_wav)

with open('flac/2.flac', 'rb') as f_flac:

    x_flac, rate = sf.read(f_flac)

#exchange sound formats by saving

sf.write('1.flac', x_wav, 16000)

sf.write('2.wav', x_flac, 16000)

Python - Create an empty array and apply maximum/minimum clipping limits to a vector

 The code below shows how to

1. Define array a as type = float 32. (aka Single in Matlab)

2. Create empty array of zeros b as the same shape and type as a

3. Apply min/max limits to a and the clipped results to b

4. Make a copy of a as c and apply the same min/max clipping limits.

import numpy as np

#Array of float32. Without dtype, the default is float64.

a = np.array([0.21, 0.92, 0.89, 1.67, -0.12],dtype=np.float32)

#Create b with the same dimension as a

b = np.zeros(a.shape,dtype=a.dtype)

#clip

np.clip(a,0,1,out=b);

#make a copy of a as c. If use = c, the content of a will change with c.

c = a.copy()

c[c>1]=1

c[c<0]=0

Variable explorer results with Spyder:

Reference:

Single-precision floating-point (Wikipedia)

Python and Openpyxl - Read in multiple txt files and store as different worksheets in a single Excel file

If you have some text file and want to combine them into different worksheets of the same Excel file, simply follow the example below using Python:

1. Create some txt files and separate data with tabs:

2. Save the following code as a python file:

import csv
import openpyxl

inputArray = ['a', 'b', 'c']

wb = openpyxl.Workbook()
output_file = 'output.xlsx'

#loop in the reversed order so that the first file is stored as the first sheet.
for item in reversed(inputArray):

    input_file = item+'.txt'
    sheet = wb.create_sheet(item, 0)

    with open(input_file, 'rt') as data:
    reader = csv.reader(data, delimiter='\t')
    for row in reader:
    sheet.append(row)

wb.save(output_file)

Note: install openpyxl using pip or miniconda if required.

3. result:

Reference:

如何使用 Python 程式操作 Excel 試算表 (TechBridge 技術共筆部落格 by kdchang)

Raspberry - Pronunciation 什麼!? 樹莓派的英文發音竟然跟你想的不一樣？

你接觸樹莓派(Raspberry Pi)多久了呢？

「樹莓」的英文字Raspberry看起來不難發音吧... 你也是唸Rasp+Berry嗎？

其實，跟想像的不一樣喔！竟然是：

p不發音！p不發音！p不發音！(很重要所以說三次！)


所以唸起來比較像是Raz-berry

記起來了嗎？

發音聽這邊：

How to say raspberry (Cambridge Dictionary YouTube) (包括英式與美式發音)

這30個台灣人常唸錯的英文字😲 你答對幾個？ (Lily Chen YouTube 6:00)

JavaScript and JSON

If we have a JSON string like this:

{"result":{"information":[{"name":"Tom","age":25},{"name":"Andy","age":30},{"name":"Peter","age":35},{"name":"David","age":40}]}}

How do we use it in JavaScript?

The answer is very easy. Just use JSON.parse().

JavaScript code:


<script language="JavaScript">
<!--

var JSONStr ='{"result":{"information":[{"name":"Tom","age":25},{"name":"Andy","age":30},{"name":"Peter","age":35},{"name":"David","age":40}]}}';
var JSONObj = JSON.parse(JSONStr);

var count;
for (count = 0 ; count <=3 ; count++){
    document.write("<br><br>");
    document.write(JSONObj.result.information[count].name+" ");
    document.write(JSONObj.result.information[count].age);
}
</script><noscript>
Your browser doesn't support JavaScript.
</noscript>

Result:


Related Information:

Simple JavaScript code - Hello World and Current Time
iOS: Print out JSON content in Swift playground

Simple JavaScript code - Hello World and Current Time

Here is a simple JavaScript code for beginners:

<script language="JavaScript">
<!--
//Hello, World and Random Number
document.write("Hello, JavaScript!");
var myRand = Math.round(Math.random()*100);
var myStr1 = "<br><br>My Random number between 0 and 100:  <b><font size=+2 color=red>"
var myStr2 = "</font></b><br><br>"
document.write(myStr1+myRand+myStr2);

//Date and Time
var month_array = new Array("January", "February", "March", "April", "May", "June","July", "August", "September", "October", "November", "December");
var weekday_array = new Array("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday");

var nowDate = new Date();
var year = nowDate.getFullYear();
var month = nowDate.getMonth();
var day = nowDate.getDate();
var weekday = nowDate.getDay();
var hour = nowDate.getHours();
var ampmStr = hour > 12 ? " PM" : " AM";
hour %= 12;
var minute = nowDate.getMinutes();
var second = nowDate.getSeconds();

//Add 0 if only one digit. See: http://stackoverflow.com/questions/8513032/less-than-10-add-0-to-number
hour = ('0' + hour).slice(-2)
minute = ('0' + minute).slice(-2)
second = ('0' + second).slice(-2)

document.write("Now is:<br>");
document.write(hour+":"+minute+":"+second+ampmStr+"<br>")
document.write(weekday_array[0]+"<br>"+month_array[month]+" "+day+", "+year+"<br><br>")


//Simple Calculation
var a = 3;
var b = 4;
var sum = a+b;
document.write(a+" + "+b+" = "+sum)
</script>
<noscript>
Your browser doesn't support JavaScript.
</noscript>

Result:



Related Information:

JavaScript and JSON

Sending RSA encrypted message - From Python socket server to iOS device (Raspberry Pi Part)

Secure Data Transmission: Raspberry Pi (Python) -> iOS device (Swift)

Secure communication between a mobile device client and a server is important. This tutorial selects an iPhone as the server and uses a Raspberry Pi as the client. The communication procedure is as below:

1. iPhone generates an RSA private / public key pair and sends the public key to server.
2. Server encrypts a message using the public key provided by iPhone and sends the encrypted message to iPhone
3. iPhone decrypts the encrypted message from server using the private key

This tutorial is derived from the basic communication between an iPhone and a Python socket server built on a Raspberry Pi without encryption:

Connect an iPhone to a Simple Python Socket Server (Raspberry Pi Part) (iOS Part)

For an overview on client-server communications, see this:

Communication between iOS device (Client) and Raspberry Pi (Server)

Raspberry Pi Part

Raspberry Pi 2 Model B is used. The Python version used is 2.7.3. Check the Python version with the python -V command.

1. Install Python-Crypto.

sudo apt-get install python-crypto

2. Use sudo nano server.py command to edit a python file as below:

import socket

from Crypto.PublicKey import RSA

from Crypto.Cipher import PKCS1_v1_5

#Declartion

mysocket = socket.socket()

host = socket.gethostbyname(socket.getfqdn())

port = 7777

publicKeyType_str = "public_key="

message = "Secret message from RPi server"

if host == "127.0.1.1":

    import commands

    host = commands.getoutput("hostname -I")

print "host = " + host

#Prevent socket.error: [Errno 98] Address already in use

mysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

mysocket.bind((host, port))

mysocket.listen(5)

c, addr = mysocket.accept()

while True:

    #Wait until data is received.

    data = c.recv(1024)

    data = data.replace("\r\n", '') #remove new line character

    if publicKeyType_str in data:

        #remove publicKey_str

        data = data.replace(publicKeyType_str, '')

        #convert string to key

        public_key = RSA.importKey(data)

        #add PKCS1 v1.5 padding

        public_key_padding = PKCS1_v1_5.new(public_key)

        #print(public_key.exportKey()) #Check this with Xcode's debug console.

        print("Client's public key received!")

        #encrypt

        encrypted = public_key_padding.encrypt(message)

        c.send(encrypted)

        print "Encrypted message sent!"

    elif data == "Quit": break

#Server to stop

print "Server stopped"

c.close()

iOS Part

Write an iOS app in Swift. See the client version of this post:

Sending RSA encrypted message - From Python socket server to iOS device (iOS Part)

Result

Run both the socket server on Raspberry Pi and iOS simulator on Mac. Press the four app buttons from top to bottom. Raspberry Pi sends "Secret message from RPi server" to iPhone:

Go back to Communication between iOS device (Client) and Raspberry Pi (Server)

Sending RSA encrypted message - From iOS device to Python socket server (Raspberry Pi Part)

Secure Data Transmission: iOS device (Swift) -> Raspberry Pi (Python)

Secure communication between a mobile device client and a server is important. This tutorial selects a Raspberry Pi as the server and uses an iPhone simulator as the client. The communication procedure is as below:

1. Server generates an RSA private / public key pair and sends the public key to iPhone
2. iPhone encrypts a message using the public key provided by server and sends the encrypted message to server
3. Server decrypts the encrypted message from iPhone using the private key

This tutorial is derived from the basic communication between an iPhone and a Python socket server built on a Raspberry Pi without encryption:

Connect an iPhone to a Simple Python Socket Server (Raspberry Pi Part) (iOS Part)

For an overview on client-server communications, see this:

Communication between iOS device (Client) and Raspberry Pi (Server)

Raspberry Pi Part

Raspberry Pi 2 Model B is used. The Python version used is 2.7.3. Check the Python version with the python -V command.

1. Install Python-Crypto.

sudo apt-get install python-crypto

2. Use sudo nano server.py command to edit a python file as below:

import socket

from Crypto.PublicKey import RSA

from Crypto import Random

#Generate private key and public key

random_generator = Random.new().read

private_key = RSA.generate(1024, random_generator)

public_key = private_key.publickey()

#Declartion

mysocket = socket.socket()

host = socket.gethostbyname(socket.getfqdn())

port = xxxx

encrypt_str = "encrypted_message="

if host == "127.0.1.1":

    import commands

    host = commands.getoutput("hostname -I")

print "host = " + host

#Prevent socket.error: [Errno 98] Address already in use

mysocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

mysocket.bind((host, port))

mysocket.listen(5)

c, addr = mysocket.accept()

while True:

    #Wait until data is received.

    data = c.recv(1024)

    data = data.replace("\r\n", '') #remove new line character

    if data == "Client: OK":

        c.send("public_key=" + public_key.exportKey() + "\n")

        print "public key sent"

    elif encrypt_str in data:

        #remove encrypt_str

        data = data.replace(encrypt_str, '')

        #decrypt

        decrypted = private_key.decrypt(data)

        #remove padding

        #See Link: How to decrypt an RSA encrypted file in Python

        if len(decrypted) > 0 and decrypted[0] == '\x02':

            pos = decrypted.find('\x00')

            if pos > 0:

                c.send("Server: OK")

                message = decrypted[pos+1:]

                print message

    elif data == "Quit": break

#Server to stop

c.send("Server stopped\n")

print "Server stopped"

c.close()

iOS Part

Write an iOS app in Swift. See the client version of this post:

Sending RSA encrypted message - From iOS device to Python socket server (iOS Part)

Result

Run both the socket server on Raspberry Pi and iOS simulator on Mac. Press the four app buttons from top to bottom. iPhone sends "Secret message from iPhone!!" to Raspberry Pi:

Reference:

How to decrypt an RSA encrypted file in Python

Go back to Communication between iOS device (Client) and Raspberry Pi (Server)

Raspberry Pi Setup Procedure

Due to the boot failure of my Raspberry Pi 2 Model B, I tried to make a record of the installation procedure as below:

1. Purchase a Raspberry Pi and Install Raspbian.

2. Configure keyboard as American (US) layout.

3. Connect to Wi-Fi.

4. Update and upgrade with these commands:

sudo apt-get update

sudo apt-get upgrade

5. Configure the time and date.

6. Enable remote access with SSH terminal command if required.

7. Install the VNC server if remote desktop is required. (Useful for screen capture)

8. Install SAMBA server if file exchange is required.

9. Install Apache HTTP server.

10. Install PHP. How to create a PHP file: PHP Basics.

11. Install MySQL database server.

Note: Linux + Apache + MySQL + PHP is known as LAMP.

12. Install the Vim text editor if required.

13. Install FTP server if required.

14. Install more servers if required.

=======
Chinese version of this post (含中文字型安裝)

Getting Started with Raspberry Pi

This post is for people who just bought a Raspberry Pi.

Requirements

- You need to purchase a Raspberry Pi first.

- You may also need:

PRi case

Power supply with Micro USB connector

Micro SD card

(At least one SD adapter is required to be inserted into a computer.)

(If you want to switch between different operating systems,

you may buy 2 or more microSD cards.)

An HDMI-to-DVI cable

(The monitor needs to have a DVI-D input connector.)

Wi-Fi USB dongle (for Raspberry Pi 2 Model B)

(Raspberry Pi 3 includes built-in Wi-Fi function and hence doesn't need it.)

My Wi-Fi USB dongle is EDUP EP-N8508GS

You may choose any one of compatible USB Wi-Fi adapters.

Put the Raspberry Pi in a case.

If you connect the RPi to a monitor and a power supply now, nothing will happen.

This is because the microSD card is required.

A newly purchased microSD card should have already been formatted.

Go to the official Raspberry Pi website and download the NOOBS installer.

Unzip the file to the SD memory card, which is then insert into the RPi.

Connect RPi with a mouse and a keyboard, then start booting!

If you are using a Mac to copy NOOBS and unable to write to the memory card

because it is 'read-only',

try adjust the switch of the adapter to the middle as this video:

MacBook Pro SD Card Read Only Problem Solved

I tried it once. It did work!

Connect RPi to power.

Select the first option Raspbian, which is the official operating system

Installing...

Remember the default username is pi, and the password is raspberry.

If you don't want to change any settings, select Finish.

If the window/graphical interface is not displayed, type the startx command.

Connect the RPi to a network cable, and you should be able to surf the internet with the Epiphany web browser!!

========

Default Login ID and password:
raspberrypi login: pi
Password: raspberry

Basic commands:

Start the graphical user interface (X Window)
startx

Run commands as the superuser:
sudo

Show directories and files at the current path:
ls

Show the current path:
pwd

Show network information such as IP address:
ifconfig
hostname -I

Show the remaining space of the microSD card:
df -h /dev/root

Show the Linux kernel version:
uname -a

Clear the screen:
clear

Shut down the RPi:
sudo halt

Reboot the RPi:
sudo reboot

Create a folder:
mkdir folder_name

Enter a folder:
cd folder_name

Exit a folder:
cd ..

Return to the home directory of user pi:
cd

Commands to update and upgrade RPi:

sudo apt-get update

sudo apt-get upgrade

A good function with the Tab key:

While typing commands, hit the tab key to autocomplete the name of a file/folder.

For example, type cd De. Then hit the tab key, the command line becomes:
cd Desktop/

=====
Go back to Raspberry Pi Setup Procedure.
Getting Started 與樹莓派的初次接觸 (Chinese version of this post.)

Using OpenSSL to decrypt an encrypted message exported by Python

This tutorial involves three parts:

(1) OpenSSL: Generate a private and public key pair using. 
(2) Python: Use the public key to export an encrypted message.
(3) OpenSSL: Decrypt the message using the private key.

(1) OpenSSL

Generate an RSA private and public key pair in PEM format.

1. Generate a 1024-bit private key:

sudo openssl genrsa -out private_key.pem 1024

2. Obtain a public key from the private key:

sudo openssl rsa -in private_key.pem -pubout -out public_key.pem

(2) Python

1. Edit a Python file called encrypt.py as:

from Crypto.PublicKey import RSA

from Crypto.Cipher import PKCS1_v1_5

f = open("public_key.pem","r")

public_key = RSA.importKey(f.read())

f.close()

# Generate a cypher using the PKCS1 v1.5 standard.
# See: GitHub: EncryptionExample/python/encrypt.py

cipher = PKCS1_v1_5.new(public_key)

message = "Secret message in Python file"

encrypted = cipher.encrypt(message)

f = open("encrypted.txt","w")

f.write(encrypted)

f.close()

print "OK"

2. Produce the  file with this command:

sudo python encrypt.py

(3) OpenSSL


Decrypt encrypted.txt with private_key.pem using this command:

openssl rsautl -in encrypted.txt -decrypt -inkey private_key.pem

Note


PKCS#1 v1.5 padding is required in encrypt.py. Without it, this error may happen with the OpenSSL decryption command:

RSA operation error

rsa routines:RSA_EAY_PRIVATE_DECRYPT:data greater than mod len:rsa_eay.c:523:

References:

Encrypt/decrypt a string with code-generated RSA public/private keys in Python
Encrypt/decrypt a string with RSA public/private PEM files using Python
OpenSSL RSA commands to encrypt/decrypt a message in terminal

Read/Write a file in Python

The Python code below shows how to write a string to a storage file and then print the file content by reading:

open("storage.txt","w").write("Hello, World!")

readStr = open("storage.txt","r").read()

print readStr

iOS version:

Read/Write a file in an iOS app (Swift 2)

OpenSSL RSA commands to encrypt/decrypt a message in terminal

It is possible to use OpenSSL commands to:

(1) generate an RSA private/public key pair.
(2) encrypt and decrypt a message using the private/public keys generated.

Some explanations:
(1) Generate an RSA private key first. The public key can then be obtained from the private key.
(2) Use different keys of the same key pair for encrypting/decrypting. For example, encrypt a message with the public key and then decrypt the encrypted message with the private A cryptosystem using this public-private key mechanism is known as asymmetric because different keys are used for encrypting/decrypting.

The following terminal commands have been tested on a Raspberry Pi and a Mac.

For Raspberry Pi, commands may need an extra sudo word in front of openssl or executed in the desktop folder.

1. Generate an RSA private and public key pair in PEM format:

Generate a 1024-bit private key:

openssl genrsa -out private_key.pem 1024

Obtain a public key from the private key:

openssl rsa -in private_key.pem -pubout -out public_key.pem

2. Create a message.txt file and edit its content:

sudo nano message.txt



3. Encrypt message.txt with public_key.pem using this command:

openssl rsautl -in message.txt -encrypt -pubin -inkey public_key.pem > encrypted.txt

4. Decrypt encrypted.txt with private_key.pem using this command:

openssl rsautl -in encrypted.txt -decrypt -inkey private_key.pem

References:

Raspberry Pi / Linux:
Encrypt/decrypt a string with RSA public/private keys in PHP
Encrypt/decrypt a string with RSA public/private PEM files using Python
Sending an RSA encrypted message from client to Python socket server

iOS:
Encrypt/decrypt a string with RSA public/private keys in Swift
Encrypt/decrypt a string with public/private keys imported from PEM files

Go back to Communication between iOS device (Client) and Raspberry Pi (Server)