Generalized Semantic Regression using Contextual Embeddings
In many applications actuaries, data scientists or researchers are confronted with datasets like shown in Table 1. While nominal variables, as in the first or second column, can be used …
Guessing Passwords using an LSTM Network
Introduction A while ago I applied NLP strategies to implement an algorithm that is capable to guess a password. Since then a new method called transformer was developed and successfully …
Building a minimal, cost efficient Dask cluster
In this article we will show a way to do high performance parallel computing at a Kubernetes cluster using task. A primary focus is that we want to archive the …
Kernel Regression using the Fast Fourier Transform
1. Setup In a previous post it was shown how to speed up the computation of a kernel density using the Fast Fourier Transform. Conceptually a kernel density is not …
Fast Kernel Density Estimation using the Fast Fourier Transform
1. Setup This Post is about how to speed up the computation kernel density estimators using the FFT (Fast Fourier Transform). Let be be a random sample drawn from an …
Non-Linear Classification Methods in Spark
In a previous post I covered how to apply classical linear estimators like support vector machines or logistic regression to a non-linear dataset using the kernel method. This article can …
Non-Linear Support Vector Machines (SVM)
1. Introduction This blog post is about Support Vector Machines (SVM), but not only about SVMs. SVMs belong to the class of classification algorithms and are used to separate one …
Kernel Regression using Pyspark
1. Kernel Regression using Pyspark In a previous article I presented an implementation of a kernel denisty estimation using pyspark. It is thus not difficult to modify the algorithm to …
Nonparametric Density estimation using Spark
1. A Nonparametric Density implementation in Spark One of my previous blog post concerns about nonparametric density estimation. In this post i presented some Matlab code. An advantage of this …
Functional Regression with Spark
1. Functional Regression Let the covariate be an at least twice continuously differentiable random function defined wlog. on an interval and the corresponding the response. For simplicity we assume centered …
Functional Principal Component Analysis with Spark
1.) Functional Principal Component Analysis Let be a centered smooth random function in , with finite second moment . Without loss of generality we assume instead of some arbitrary compact …
2. Coding the “Educated Guess Procedure”
1. Perform the Analyze To start with, we load the “rockyou.txt.tar.gz” password list using wget. I’m not sure if it is legal to provide a link for the list, therefore …
5. Running some tests
1. Test the Enviroment 1.1 Simulation of a Brownian Motion The purpose of the first notebook entry is to check if matplotlib is correctly installed. We simulate 20 Brownian Motions …