## Kernel Regression using Pyspark

1. Kernel Regression using Pyspark In a previous article I presented an implementation of a kernel denisty estimation using pyspark. It is thus not difficult to modify the algorithm to …

## Nonparametric Density estimation using Spark

1. A Nonparametric Density implementation in Spark One of my previous blog post concerns about nonparametric density estimation. In this post i presented some Matlab code. An advantage of this …

## Functional Regression with Spark

1. Functional Regression Let the covariate be an at least twice continuously differentiable random function defined wlog. on an interval and the corresponding the response. For simplicity we assume centered …

## Functional Principal Component Analysis with Spark

1.) Functional Principal Component Analysis Let be a centered smooth random function in , with finite second moment . Without loss of generality we assume instead of some arbitrary compact …

## 3. A more sophisticated approach using Markov chains.

1. Generalize the Procedure The very simple Educated Guess Procedure is not only very simple, the procedure is also very unrealistic. At most the assumption that are independent and identically …

## 2. Coding the “Educated Guess Procedure”

1. Perform the Analyze To start with, we load the “rockyou.txt.tar.gz” password list using wget. I’m not sure if it is legal to provide a link for the list, therefore …

## 5. Running some tests

1. Test the Enviroment 1.1 Simulation of a Brownian Motion The purpose of the first notebook entry is to check if matplotlib is correctly installed. We simulate 20 Brownian Motions …

## 4. Install IPython Notebook for Remote Access and Hive

1. Requirements 2. Install Software In this section we will install some stuff which will make life easier. In constrast to Spark or Hadoop it is only required to install …

## 3. Build the Cluster

1. Requirements We need an SD Card with Lubuntu, Hadoop and Spark installed. 2. Build the Cluster 2.1 Clone the SD Card sudo shutdown 0 of your orangepi and remove …

## 2. Install Hadoop and Spark

1. Requirements An Orangepi with Lubuntu running, see this post for further instructions. 2.Install the Components 2.1 Update Java In fact Hadoop is not necessary for Spark. However, we will …