CUDA Controlling NVIDIA GPU Fans on a headless Ubuntu system When performing data science on an Ubuntu Linux machine remotely, the fans on NVIDIA GPUs may not spin up in response to increased load. This is due to fact that the NVIDIA controlling software generally requires logging into the GUI Desktop. Simply forwarding the xwindow with SSH has little effect,
TensorFlow Google's 'Coral' Edge TPU Dev Boards I received a pair of Google's 'Coral' Edge TPU Dev Boards today. They have the exact same dimensions as Raspberry Pi devices, including the location of the mounting post holes. This is convenient if you have project boxes or mounting plates that you normally use for
TensorFlow Google's 'Coral' Edge TPU Accelerator Feeling like a kid on Christmas, today I finally received a couple of the Google 'Coral' Edge TPU Accelerators in the mail. First impressions upon unboxing is that they're not as physically robust as the Intel Neural Compute Stick 2. The entire Intel NCS2 case is
python Pandas - Opening and Selecting Data Data assets used in these examples * Titanic [https://www.kaggle.com/c/titanic] * City population [http://ezlocal.com/blog/post/Top-5000-US-Cities-by-Population.aspx] * USGS ShakeMap Atlas [https://earthquake.usgs.gov/data/shakemap/] -------------------------------------------------------------------------------- Reading data from CSV files Reading CSV files and using one of the columns
spark Connect Jupyter to Remote Spark Clusters With Apache Toree Scala [https://www.scala-lang.org/] is a fun language which gives you all the power of Java [https://www.java.com], with the simplicity of Python [https://www.python.org/], and the power of functional programming [https://en.wikipedia.org/wiki/Functional_programming]. Though there are a variety of
python How to run TensorFlow with GPU on Windows 10 in a Jupyter Notebook Install CUDA ToolKit The first step in our process is to install the CUDA ToolKit, which is what gives us the ability to run against the the GPU CUDA cores. Because TensorFlow [https://www.tensorflow.org/] is very version specific, you'll have to go to the CUDA ToolKit
spark Transpose data with Spark A short user defined function written in Scala which allows you to transpose a dataframe without performing aggregation functions.
spark Convert Spark Vectors to DataFrame Columns Vectors are typically required for Machine Learning tasks, but are otherwise not commonly used. Sometimes you end up with an assembled Vector [https://spark.apache.org/docs/latest/ml-features.html#vectorassembler] that you just want to disassemble into its individual component columns so you can do some Spark SQL
spark Pivoting data with Spark One of the common data engineering tasks is taking a deep dataset and turning into a wide dataset with some sort of aggregation function. Let's take a quick look at an example dataset to see why we would want to perform this action. Our goal: To determine if
spark Renaming All Columns In A Spark DataFrame Here's an easy example of how to rename all columns in an Apache Spark DataFrame. Tehcnically, we're really creating a second DataFrame with the correct names. // IMPORT DEPENDENCIES import org.apache.spark.sql.SparkSession import org.apache.spark.sql.functions._ import org.apache.spark.sql.{SQLContext,
spark Featured Using Spark, Scala and XGBoost On The Titanic Dataset from Kaggle The Titanic: Machine Learning from Disaster [https://www.kaggle.com/c/titanic] competition on Kaggle [https://www.kaggle.com/] is an excellent resource for anyone wanting to dive into Machine Learning [https://en.wikipedia.org/wiki/Machine_learning]. There are forums [https://www.kaggle.com/c/titanic/discussion] where you
africa Family of Elephants in the Mara Copyright: James Conner Camera: Canon EOS 5D Mark III Lens: Canon 100-400L F4 with 1.4x Internal Extender Stats: 560mm/ƒ/5.6/1/1600s/ISO 640 Taken: March 5 2015
spark List All Additional Jars Loaded in Spark Once in a while, you need to verify the versions of your jars which have been loaded into your Spark session. Fortunately, there's a relatively easy way to do this: the listJars method. As you can see from the example below, the listJars method shows all jars loaded
spark Spark Vector of Vectors I recently ran into a problem with creating a features vector for a machine learning project. If the number of features in your dataframe is too large, the JVM will crash during the Catalyst optimizer process because the number of constant variables generated exceeds the JVM limit of 65,536.
spark Joining Spark DataFrames Without Duplicate or Ambiguous Column Names When performing joins in Spark, one question keeps coming up: When joining multiple dataframes, how do you prevent ambiguous column name errors? 1) Let's start off by preparing a couple of simple example dataframes // Create first example dataframe val firstDF = spark.createDataFrame(Seq( (1, 1, 2, 3, 8,
spark Selecting Dynamic Columns In Spark DataFrames (aka Excluding Columns) I often need to perform an inverse selection of columns in a dataframe, or exclude some columns from a query. This is a very easy method, and I use it frequently when arranging features into vectors for machine learning tasks. import org.apache.spark.sql.Column // Create an example dataframe
photography Photography laser trigger trap I've decided to try my hand at droplet and splash photography. Rather than using a manual method (setting up a constant drip method and trying to snap a picture the moment a droplet lands), I decided to go the engineered route (surprise!). I did a lot of searching
photography Cheetah in tall grass in the Maasai Mara Copyright: James Conner Camera: Canon EOS 5D Mark III Lens: Canon 100-400L F4 with 1.4x Internal Extender Stats: 560mm/ƒ/5.6/1/1600s/ISO 640 Taken: March 5 2015
photography Intent lion cub in the Maasai Mara Copyright: James Conner Camera: Canon EOS 5D Mark III Lens: Canon 100-400L F4 with 1.4x Internal Extender Stats: 366mm/ƒ/5.6/1/400s/ISO 400 Taken: February 28 2015
spark Joining Data Frames in Spark SQL Data The data that I'm using for this test comes from Kaggle's [https://www.kaggle.com/] Titanic Project [https://www.kaggle.com/c/titanic]. The purpose of the Titanic project is to create a machine learning model to predict the survivability of the Titanic passengers. In
spark Create an Apache Spark cluster using Raspberry Pi 2 Nodes Everyone talks about Spark in the context of "Big Data", but I've been having fun with it on a tiny platform: Raspberry Pi 2 (RP2) micro computers! Though the specs of the RP2 are modest, you can easily run Spark on a single node ... but where&
spark Spark Word Count In the world of Big Data, your first program isn't "Hello, World!", it's Word Count, because it's easy to understand and conceptualize. In Java MapReduce, this would take about 4 pages of code. In Scala/Spark, it can be done in a
