Virtual Artifact

Coming soon

This is Virtual Artifact, a brand new site by Froilan Miranda that's just getting started. Things will be up and running here shortly, but you can subscribe in the meantime if you'd like to stay up to date and receive emails when new content is published!

AWS API Gateway Introduction

Today API's are the hot thing, and for good reason. The add simplicity and flexibility to complex data architectures. API Gateway offers the ability to create RESTful APIs and WebSocket APIs within the AWS Cloud Platform. API Gateway also supports serverless, containerized workloads. Amazon API Gateway is an AWS service

AWS CodeCommit Introduction - Part 2

In the previous article we looked at a general overview of the AWS CodeCommit service. In this article we will look at setup and accessing a repository in CodeCommit. There are several ways to work with CodeCommit: * AWS Management Console * Use Git credentials with HTTPs * Federated Access * Temporary credentials * Web

AWS CodeCommit Introduction - Part 1

In todays development environment, Git-based services are a norm. With GitHub being the industry standard and others like Bitbucket closly following. AWS has also added there solution for Git-based version control called CodeCommit. We will look at some of the features of CodeCommit and what makes it special compared to

Getting Started

PySpark - Spark SQL Context

Spark aims to make it easy to work with data. One way they achieve this is by working with spark data as if you were working on a SQL database * Spark SQL enables querying of DataFrames as database tables * Temporary per-session and global tables * The Catalyst optimizer makes SQL queries

PySpark - Using DataFrames

Spark DataFrames Previously we looked at RDDs, and were the primary data set in Spark 1. In Spark 2 we rarely use RDDs only for low level transformations and control over the dataset. If the data is unstructured or streaming data we then have to rely on RDDs, for everything

PySpark - Using RDDs

I will keep my data in a folder named 'datasets' If PySpark is not already loaded up, go ahead and start PySpark and create a new Jupyter notebook View information about the SparkContext by inputing sc If we were running a cluster of nodes the output would be a bit

Getting Started

Spark 2 setup

Demo will cover: * Install standalone Spark on your local machine * Set up the PySpark REPL interface Req's * This demo will by done with Python 3 * Java v8 * jupyter notebooks Download Spark 2 from https://spark.apache.org/downloads.html 1. Choose the most recent 2.x build 2. Choose package:

AWS - Creating a simple VPC and EC2 instance

Introduction This article with step through the process of quickly creating a VPC and attching an EC2 instance Creating a VPC Rescource by region will automaticlly have a default vpc, subnets, route tables and more created for you. One default for every Region. We can use the default VPC and

Pandas and MySql with a hit of AWS RDS

Introduction This article will look at connecting Python and MySQL database. With the help of some sql connection tools, transfering data between Python and MySQL will be simplified. Finally, we will migrate and run the database on a cloud platform prerequisites * database with some data to work on * aws developer

Creating a AWS S3 service with Python

Overview In a previous module we use the boto3 library to connect our python script with an AWS service. Here, we are going continue down this path by taking a look at some operations we can perform using AWS S3 and python. We will create a storage service with python

Using AWS-CLI to interact with AWS S3

Overview This article discusses some of the common commands of AWS-CLI to communicate with the AWS S3 service. The command line tool is a quick and easy way to manage S3 buckets. It is not a complicate interface, with only a hand full of commands. The following will use these

Install and Configure Boto3 with AWS

Introduction AWS (Amazon Web Services) is an ecosystem with an abundace of services to fufill many of our development needs. There are also some great ways for us to interact with these services. We can simply log in to the web browser console(https://aws.amazon.com ). Or we can

AWS - Install and Setup Command Line Interface

Overview Amazon Web Services offer a large number of cloud based services and tools. There are a few different ways that we can interact with these services. These options include a web browser interface (https://aws.amazon.com/) and SDKs (Software Development Kits) for most major programming language. This article

ITerm 2 Cheat Sheet

Tabs and Windows Function Shortcut New Tab ⌘ + T Close Tab or Window ⌘ + W (same as many mac apps) Go to Tab ⌘ + Number Key (ie: ⌘2 is 2nd tab) Go to Split Pane by Direction ⌘ + Option + Arrow Key Cycle iTerm Windows ⌘ + backtick (true of all mac apps and works with desktops/

IntelliJ IDEA Cheat Sheet

Create a class : Alt+Home (Alt-Fn-left arrow) and then Ctrl-N

Markdown Cheat Sheet

Below are some resources: * https://www.markdownguide.org/cheat-sheet/ * https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet * https://guides.github.com/pdfs/markdown-cheatsheet-online.pdf

Nginx

A page of useful commands # controlling nginx service sudo service nginx start sudo service nginx stop sudo service nginx restart