How to create a H2O cluster on google cloud with static external IP?
Creating a new H2O-3 Cluster deployment in google cloud gives only 2 options for reserving an IP address: Ephemeral and None. Is it possible to create a h2o cluster with a static IP address. Using the "addresses" command in gcloud like:
gcloud compute addresses create h2oflow --addresses /* ephemeral external IP assigned to h2o cluster */
assigns the h2oflow address to only 1 of the nodes in the cluster. Is it possible to assign a static IP to an entire cluster of h2o nodes?
See also questions close to this topic
Failing to visualize quaternions in R with rotations package
I have to analyze a huge dataframe containg quaternions, that needs to be analyzed. I decided to use R in order to visualize my dataset using the
Unfortunately, I was unable to even visualize just one quaternion. Instead I'm obtaining obscure errors.
Here is my code:
require(rotations) q<-as.Q4(c(1,0,0,0)) plot(q)
Which issues the following warning after trying to plot:
Error: Aesthetics must be either length 1 or the same as the data (22006): alpha, x, y
What I'm doing wrong here?
R version 3.4.3 (2017-11-30) Platform: i386-w64-mingw32/i386 (32-bit) Running under: Windows 7 x64 (build 7601) Service Pack 1 Matrix products: default locale:  LC_COLLATE=German_Germany.1252 LC_CTYPE=German_Germany.1252  LC_MONETARY=German_Germany.1252 LC_NUMERIC=C  LC_TIME=German_Germany.1252 attached base packages:  stats graphics grDevices utils datasets methods base other attached packages:  rotations_1.5 rgl_0.99.16 Rcpp_0.12.14 ggplot2_2.2.1 loaded via a namespace (and not attached):  knitr_1.20 magrittr_1.5 munsell_0.4.3  colorspace_1.3-2 xtable_1.8-3 R6_2.2.2  rlang_0.1.6 plyr_1.8.4 tools_3.4.3  webshot_0.5.1 grid_3.4.3 gtable_0.2.0  miniUI_0.1.1.1 htmltools_0.3.6 crosstalk_1.0.0  lazyeval_0.2.1 digest_0.6.13 tibble_1.4.1  shiny_1.2.0 later_0.7.5 htmlwidgets_1.3  promises_1.0.1 manipulateWidget_0.10.0 mime_0.6  compiler_3.4.3 pillar_1.1.0 scales_0.5.0  sphereplot_1.5 jsonlite_1.5 httpuv_1.4.5
align left figure caption in R markdown
figure captions in R markdown to PDF are centered by default. here's an example: with this in the YAML:
output: pdf_document: keep_tex: true number_sections: false df_print: kable latex_engine: lualatex includes: in_header: input/mystyles.sty
![Profile REU15-201](input/profile pictures/REU15-201)
any ideas how to align the caption left?
Replace na´s in specific columns by the median of the same columns
I would like to replace the na´s in v1 to v4 with the median of the same columns
Here are some example data
id <- c(1,2,3,4) v1 <- c(1,3,0,2) v2 <- c(NA,1,NA,2) v3 <- c(2,4,1,2) v4 <- c(NA,1,0,2) v5 <- c(5,1,NA,2) v6 <- c(7,1,9,NA) df <- data.frame(id, v1, v2, v3,v4,v5,v6) df_pre <- df %>% group_by(id) %>% mutate(Median_v1_v4 = median(c(v1,v2,v3,v4), na.rm=TRUE))
This is what data looks like now:
id v1 v2 v3 v4 v5 v6 Median_v1_v4 1 1 NA 2 NA 5 7 1.5 2 3 1 4 1 1 1 2.0 3 0 NA 1 0 NA 9 0.0 4 2 2 2 2 2 NA 2.0
This is what i want the data to look like
id v1 v2 v3 v4 v5 v6 Median_v1_v4 1 1 1.5 2 1.5 5 7 1.5 2 3 1.0 4 1.0 1 1 2.0 3 0 0.0 1 0.0 NA 9 0.0 4 2 2.0 2 2.0 2 NA 2.0
How can I list and delete GCP AppEngine images from the command line?
I'd like to delete my old unused AppEngine images from Google Cloud Platform, so that I'm not charged for storing them.
I can manually list and delete the images created for my GCP AppEngine project from this URL: https://console.cloud.google.com/gcr/images/GOOGLE_CLOUD_PROJECT_ID
(Obviously, replace GOOGLE_CLOUD_PROJECT_ID with an appropriate GCP project id.)
Is there a way to list them from the command line? e.g. via
This doesn't work as I would expect:
$ gcloud compute images list --no-standard-images Listed 0 items.
Neither does this:
$ gcloud container images list Listed 0 items. Only listing images in gcr.io/GOOGLE_CLOUD_PROJECT_ID. Use --repository to list images in other repositories.
It's a little painful to go in and delete lots of these manually since each image under https://console.cloud.google.com/gcr/images/GOOGLE_CLOUD_PROJECT_ID/US/appengine is in a separate directory that I have to first click into to select the image and then click the delete button, and then go back out to the appengine directory and start the process again for any other images.
Hosting an OAuth2 API on Kubernetes with ingress-nginx LoadBalancer problem
Hosting an OAuth2 API on Kubernetes (gcloud) with ingress-nginx LoadBalancer strip "Authorization" header. Why ? and how to fix this ? thank you
gcloud does not skip root level files
I am trying to deploy an SPA to GAE. I am following the configuration sample as given in their static website example. My project folder structure is as follows:
D:\Projects\Proj1 |-node_modules |-src |-www |-.babelrc |-.gitignore |-app.yaml |-package.json |-package-lock.json
and my app.yaml is as follows:
runtime: php55 service: frontend api_version: 1 threadsafe: true handlers: - url: / static_files: www/index.html upload: www/index.html - url: /(.*) static_files: www/\1 upload: www/(.*)
The project output is built to the
gcloudis used for deploying using
gcloud app deploy app.yaml --quiet --version %DOC_VERSION% --project %GCP_PROJECT_ID%command from a batch file.
I was expecting
gcloudto copy and deploy files that were only under
wwwfolder, but instead it started copying aprox 10K files which were under the
node_modulesfolder as well (I terminated the process).
So, I changed the
app.yamland now it looks like this:
runtime: php55 service: frontend api_version: 1 threadsafe: true skip_files: - ^node_modules$ - ^src$ - ^assets$ - ^\. handlers: - url: / static_files: www/index.html upload: www/index.html - url: /(.*) static_files: www/\1 upload: www/(.*)
And works as expected.
My understanding was that the
static_filessetting was used by the
gcloudtool for uploading relevant files. So, why was it uploading the files from non-mentioned folder? Is mentioning
skip_filesis the only way to properly address this behavior?
Memory Utilization R h2o
I'm working on the Titanic. However I've run into a problem where the model no longer runs but instead throws back an error that seems to have something to do with memory allocation.
Error: water.exceptions.H2OModelBuilderIllegalArgumentException: Illegal argument(s) for DRF model: DRF_model_R_1542172359909_24373_cv_1. Details: ERRR on field: _ntrees: The tree model will not fit in the driver node's memory (2.4 KB per tree x 500 > Zero ) - try decreasing ntrees and/or max_depth or increasing min_rows!
Here is my code sample:
y<-"Survived" x<-setdiff(names(newtrain_imp),y) rf_mod<-h2o.randomForest(x,y,train_set, nfolds = 10, keep_cross_validation_predictions = T, seed=233, mtries = 8,#sampling default-1 set to al ntrees = 500, max_depth = 12, validation_frame = validate_set, binomial_double_trees = T)
Classification Scores differ between H2O4GPU and Scikit-Learn
I've begun evaluating a random forest classifier using precision and recall. However, despite the train and test sets being identical for the CPU and GPU implementations of the classifier, I'm seeing differences in the returned evaluation scores. Is this a known bug within the library by chance?
Both code samples are below for reference.
from sklearn.metrics import recall_score, precision_score from sklearn.ensemble import RandomForestClassifier rf_cpu = RandomForestClassifier(n_estimators=5000, n_jobs=-1) rf_cpu.fit(X_train, y_train) rf_cpu_pred = clf.predict(X_test) recall_score(rf_cpu_pred, y_test) precision_score(rf_cpu_pred, y_test) CPU Recall: 0.807186 CPU Precision: 0.82095
from h2o4gpu.metrics import recall_score, precision_score from h2o4gpu import RandomForestClassifier rf_gpu = RandomForestClassifier(n_estimators=5000, n_gpus=1) rf_gpu.fit(X_train, y_train) rf_gpu_pred = clf.predict(X_test) recall_score(rf_gpu_pred, y_test) precision_score(rf_gpu_pred, y_test) GPU Recall: 0.714286 GPU Precision: 0.809988
How to import an excel file with h2o
I am trying to import an excel file with h2o, but apparently it only works with .csv files, there is some other function besides this for .xls files. Is it possible to load files with this format ?.
library(xlsx) write.xlsx(x = iris, file = "C:/Users/USER/Desktop/iris.xls", row.names = FALSE) write.csv(x = iris, file = "C:/Users/USER/Desktop/iris.csv", row.names = FALSE) library(h2o) h2o.init() h2o.iris <- h2o.importFile(path = "C:/Users/USER/Desktop/iris.xls") #h2o.iris <- h2o.importFile(path = "C:/Users/USER/Desktop/iris.csv")