Onnxruntime NodeJS set intraOpNumThreads and interOpNumThreads by execution mode

See also questions close to this topic

• Error message : MongoServerError: bad auth :Authentication failed

  What is the reason behind this error? this code I am using to connect to DB.

  const uri =`mongodb+srv://${process.env.DB_USER}:${process.env.DB_PASSWORD}@cluster0.xft2s.mongodb.net/myFirstDatabase?retryWrites=true&w=majority`;
  
• In Mongo, If a document I'm saving "Prateek" then I don't want on the next create operation even the "prateek" or "praTEEK", etc is saved

  //** If I'm adding a new document with the name: "India", then I don't want that the DB allow another name with the name: "INDIA", "india", "indIA", etc. I'm new and learning, help would be great!!**

  // Controller

  var Dinosaur = require('../models/dinosaurs');
  
  //addDino
  module.exports.addDino = (req, res) => {
      var name = req.body.name;
      var type = req.body.type;
      var height = req.body.height;
      var weight = req.body.weight;
      var Period = req.body.Period;
  
      req.checkBody('name', 'Name is required').notEmpty();
  
      var errors = req.validationErrors();
      if (errors)
          return res.status(400).send({
              message: 'Name is Required'
          });
      else {
          let newDino = {
              name: name,
              type: type,
              height: height,
              weight: weight,
              Period: Period
          }
          Dinosaur.addDino(newDino, (err, result) => {
              if (err) {
                  if (err.name) return res.status(409).send({
                      message: name + ' Already Exist'
                  });
                  else if (err.url) return res.json({ status: false, error: { url: "Url already exist" }, message: err.url });
                  else return res.json(err, "Server Error");
              }
              else {
                  return res.status(200).send({
                      message: "Done"
                  });
              }
          });
      }
  }
  

  // Model

  var mongoose = require('mongoose');
  
  //dinosaur schema
  var DinosaurSchema = mongoose.Schema({
      name: {
          type: String,
          unique: true
      },
      type: {
          type: String
      },
      height: {
          type: Number
      },
      weight: {
          type: Number
      },
      Period: {
          type: String
      }
  });
  
  var Dinosaur = mongoose.model('dinosaur', DinosaurSchema);
  
  //add
  module.exports.addDino = (query, callback) => {
      Dinosaur.create(query, callback);
  }
  

  // GetAll, Already Created a new document with the name "Brachiosaurus"

  

  // > Create, a new create with the first letter lower case "brachiosaurus", Don't want it to be pushed.

  //Get All, Got pushed.
• i am trying to run the node of ganache but getting this error

  I am trying to run the node of ganache but getting this error.please resolve my problem.
• Training an ML model on two different datasets before using test data?

  So I have the task of using a CNN for facial recognition. So I am using it for the classification of faces to different classes of people, each individual person being a it's own separate class. The training data I am given is very limited - I only have one image for each class. I have 100 classes (so I have 100 images in total, one image of each person). The approach I am using is transfer learning of the GoogLenet architecture. However, instead of just training the googLenet on the images of the people I have been given, I want to first train the googLenet on a separate larger set of different face images, so that by the time I train it on the data I have been given, my model has already learnt the features it needs to be able to classify faces generally. Does this make sense/will this work? Using Matlab, as of now, I have changed the fully connected layer and the classification layer to train it on the Yale Face database, which consists of 15 classes. I achieved a 91% validation accuracy using this database. Now I want to retrain this saved model on my provided data (100 classes with one image each). What would I have to do to this now saved model to be able to train it on this new dataset without losing the features it has learned from training it on the yale database? Do I just change the last fully connected and classification layer again and retrain? Will this be pointless and mean I just lose all of the progress from before? i.e will it make new weights from scratch or will it use the previously learned weights to train even better to my new dataset? Or should I train the model with my training data and the yale database all at once? I have a separate set of test data provided for me which I do not have the labels for, and this is what is used to test the final model on and give me my score/grade. Please help me understand if what I'm saying is viable or if it's nonsense, I'm confused so I would appreciate being pointed in the right direction.
• What's the best way to select variable in random forest model?

  I am training RF models in R. What is the best way of selecting variables for my models (the datasets were pretty big, each has around 120 variables in total). I know that there is a cross-validation way of selecting variables for other classification algorithms such as KNN. Is that also a thing or if there exists a similar way for parameter tuning in RF model training?
• How would I put my own dataset into this code?

  I have been looking at a Tensorflow tutorial for unsupervised learning, and I'd like to put in my own dataset; the code currently uses the MNIST dataset. I know how to create my own datasets in Tensorflow, but I have trouble setting the code used here to my own. I am pretty new to Tensorflow, and the filepath to my dataset in my project is \data\training and \data\test-val\

  # Python ≥3.5 is required
  import sys
  assert sys.version_info >= (3, 5)
  
  # Scikit-Learn ≥0.20 is required
  import sklearn
  assert sklearn.__version__ >= "0.20"
  
  # TensorFlow ≥2.0-preview is required
  import tensorflow as tf
  from tensorflow import keras
  assert tf.__version__ >= "2.0"
  
  # Common imports
  import numpy as np
  import os
  
  (X_train_full, y_train_full), (X_test, y_test) = keras.datasets.fashion_mnist.load_data()
  X_train_full = X_train_full.astype(np.float32) / 255
  X_test = X_test.astype(np.float32) / 255
  X_train, X_valid = X_train_full[:-5000], X_train_full[-5000:]
  y_train, y_valid = y_train_full[:-5000], y_train_full[-5000:]
  
  def rounded_accuracy(y_true, y_pred):
      return keras.metrics.binary_accuracy(tf.round(y_true), tf.round(y_pred))
  
  tf.random.set_seed(42)
  np.random.seed(42)
  
  conv_encoder = keras.models.Sequential([
      keras.layers.Reshape([28, 28, 1], input_shape=[28, 28]),
      keras.layers.Conv2D(16, kernel_size=3, padding="SAME", activation="selu"),
      keras.layers.MaxPool2D(pool_size=2),
      keras.layers.Conv2D(32, kernel_size=3, padding="SAME", activation="selu"),
      keras.layers.MaxPool2D(pool_size=2),
      keras.layers.Conv2D(64, kernel_size=3, padding="SAME", activation="selu"),
      keras.layers.MaxPool2D(pool_size=2)
  ])
  conv_decoder = keras.models.Sequential([
      keras.layers.Conv2DTranspose(32, kernel_size=3, strides=2, padding="VALID", activation="selu",
                                   input_shape=[3, 3, 64]),
      keras.layers.Conv2DTranspose(16, kernel_size=3, strides=2, padding="SAME", activation="selu"),
      keras.layers.Conv2DTranspose(1, kernel_size=3, strides=2, padding="SAME", activation="sigmoid"),
      keras.layers.Reshape([28, 28])
  ])
  conv_ae = keras.models.Sequential([conv_encoder, conv_decoder])
  
  conv_ae.compile(loss="binary_crossentropy", optimizer=keras.optimizers.SGD(lr=1.0),
                  metrics=[rounded_accuracy])
  history = conv_ae.fit(X_train, X_train, epochs=5,
                        validation_data=[X_valid, X_valid])
  
  conv_encoder.summary()
  conv_decoder.summary()
  
  conv_ae.save("\models")
  

  Do note that I got this code from another StackOverflow answer.
• Print accuracy for YOLOv5 model?

  I want to print the accuracy of my trained model. I do understand when I use the model through the camera it shows me the accuracy on the box only, but I want to see the accuracy of my whole model rather than a single instance of my image.

  cap  = cv2.VideoCapture(0)
  
  # loop through labels
  for label in labels:
      print('Collecting images for {}'.format(label)) #so that we can see when transitioning through images
      time.sleep(5)  #sleep or wait for 5 seconds when transitioning between image for other class
      
      # Loop through image range
      for img_num in range(number_imgs):
          print('Collecting images for {}, image number {}'.format(label, img_num))
          
          # webcam feed
          ret, frame = cap.read()   # read the feed from the web cam and store it in given vars
          
          # Naming image path
          imgname = os.path.join(IMAGES_PATH, label+'.'+str(uuid.uuid1())+'.jpg')
          
          #Writes out image to file
          cv2.imwrite(imgname,frame)
          
          # Render to screen
          cv2.imshow('Image Collection', frame)
          
          # 2 sec delay between diff captures
          time.sleep(2)
          
          if cv2.waitKey(10) & 0xFF == ord('q'):
              break
  cap.release()
  cv2.destroyAllWindows()
  

  !cd yolov5 && python train.py --img 320 --batch 16 --epochs 500 --data dataset.yml --weights yolov5s.pt
  

  model = torch.hub.load('ultralytics/yolov5', 'custom', path='yolov5/runs/train/exp/weights/last.pt', force_reload=True)
  while cap.isOpened():
      ret, frame = cap.read()
      
      # Make detections 
      results = model(frame)
      
      cv2.imshow('YOLO', np.squeeze(results.render()))
      
      if cv2.waitKey(1) & 0xFF == ord('q'):
          break
          
  cap.release()
  cv2.destroyAllWindows()
  cv2.waitKey(1)
  
• keyerror 0 during cpu trainning

  unknow error

  I am not sure what is the reason, probably my index exceed the limit of the input? But how can I solve this error? Here is the code I used https://github.com/wanyu-lin/ICML2021-Gem
• how can I modify Dataset class to make the mask RCNN work with multiple objects?

  I am currently working on instance segmentation. I follow these two tutorials:

  1. https://haochen23.github.io/2020/06/fine-tune-mask-rcnn-pytorch.html

  2. https://colab.research.google.com/github/dlmacedo/starter-academic/blob/master/content/courses/deeplearning/notebooks/pytorch/torchvision_finetuning_instance_segmentation.ipynb#scrollTo=mTgWtixZTs3X

  However, these two tutorials work perfectly with one class like person + background. But in my case, I have two classes like a person and car + background. I didn't find any resources about making the Mask RCNN work with multiple objects.

  Notice that:

  1. I am using PyTorch ( torchvision ), torch==1.10.0+cu111 torchvision==0.11.0+cu111 torchaudio==0.10.0

  2. I am using a Pascal VOC annotation

  3. i used segmentation class (not the XML file) + the images

  and this is my dataset class

  class PennFudanDataset(torch.utils.data.Dataset):
  def __init__(self, root, transforms=None):
      self.root = root
      self.transforms = transforms
      # load all image files, sorting them to
      # ensure that they are aligned
      self.imgs = list(sorted(os.listdir(os.path.join(root, "img"))))
      self.masks = list(sorted(os.listdir(os.path.join(root, "imgMask"))))
  
  def __getitem__(self, idx):
      # load images ad masks
      img_path = os.path.join(self.root, "img", self.imgs[idx])
      mask_path = os.path.join(self.root, "imgMask", self.masks[idx])
      img = Image.open(img_path).convert("RGB")
      # note that we haven't converted the mask to RGB,
      # because each color corresponds to a different instance
      # with 0 being background
      mask = Image.open(mask_path)
  
      mask = np.array(mask)
      # instances are encoded as different colors
      obj_ids = np.unique(mask)
      # first id is the background, so remove it
      obj_ids = obj_ids[1:]
  
      # split the color-encoded mask into a set
      # of binary masks
      masks = mask == obj_ids[:, None, None]
  
      # get bounding box coordinates for each mask
      num_objs = len(obj_ids)
      boxes = []
      for i in range(num_objs):
          pos = np.where(masks[i])
          xmin = np.min(pos[1])
          xmax = np.max(pos[1])
          ymin = np.min(pos[0])
          ymax = np.max(pos[0])
          boxes.append([xmin, ymin, xmax, ymax])
  
      boxes = torch.as_tensor(boxes, dtype=torch.float32)
      # there is only one class
      labels = torch.ones((num_objs,), dtype=torch.int64) 
      masks = torch.as_tensor(masks, dtype=torch.uint8)
  
      image_id = torch.tensor([idx])
      area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0])
      # suppose all instances are not crowd
      iscrowd = torch.zeros((num_objs,), dtype=torch.int64)
  
      target = {}
      target["boxes"] = boxes
      target["labels"] = labels
      target["masks"] = masks
      target["image_id"] = image_id
      target["area"] = area
      target["iscrowd"] = iscrowd
      if self.transforms is not None:
          img, target = self.transforms(img, target)
  
      return img, target
  
  def __len__(self):
      return len(self.imgs)
  

  anyone can help me?
• Installing ncnn on Google Colab

  I have trained a custom YOLOX model on google colab and want to convert it from .onnx to .ncnn.

  I'm using the following as directions: https://github.com/Megvii-BaseDetection/YOLOX/blob/main/demo/ncnn/cpp/README.md#step4

  Step 1 requires building ncnn with directions: https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-macos

  The directions give instructions for building on different devices.

  My question: Which instructions should I use to build ncnn on Google Colab?
• yolov5 custom trained weights converted to ONNX showing wrong labels

  After converting custom trained Yolov5 weights (.pt) to ONNX and running inference on the ONNX file using:

  https://github.com/BlueMirrors/Yolov5-ONNX.git

  the detection works well but my image labels/classes are using COCO labels (ie person, airplane, etc) instead of my labels. How can I change the labels to my own. I'm unsure of the formatting of the json (or yaml?) file. Thanks!

  def detect_image(device, weight, image_path, output_image):
      # load model
      model = Yolov5Onnx(classes="coco",
                         backend="onnx",
                         weight=weight,
                         device=device)
  
      # read image
      image = cv2.imread(image_path)
  
      # inference
      preds = model(image)
      print(preds)
  
      # draw image
      preds.draw(image)
  
      # write image
      cv2.imwrite(output_image, image)
  
• How to use onnxruntime parallel with flask?

  Created a server that want to run a session of onnxruntime parallel.

  First question, will be used multi-threads or multi-processings?

  Try to use multi-threads, app.run(host='127.0.0.1', port='12345', threaded=True).
  When run 3 threads that the GPU's memory less than 8G, the program can run. But when run 4 threads that the GPU's memory will be greater than 8G, the program have error: onnxruntime::CudaCall CUBLAS failure 3: CUBLAS_STATUS_ALLOC_FAILED.

  I know that the problem is leaky of GPU's memory. But I hope that the program don't run crash. So I try to limit the number of threads, and set intra_op_num_threads = 2 or inter_op_num_threads = 2 or os.environ["OMP_NUM_THREADS"] = "2", but don't work. Try to 'gpu_mem_limit', don't work either.

  import onnxruntime as rt
  from flask import Flask, request
  app = Flask(__name__)
  
  sess = rt.InferenceSession(model_XXX, providers=['CUDAExecutionProvider'])
  
  @app.route('/algorithm', methods=['POST'])
  def parser():
      prediction = sess.run(...)
  
  if __name__ == '__main__':
      app.run(host='127.0.0.1', port='12345', threaded=True)
  

  My understanding is that the Flask HTTP server maybe use different sess for each call. How can make each call use the same session of onnxruntime?

  System information

  • OS Platform and Distribution: Windows10
  • ONNX Runtime version: 1.8
  • Python version: python 3.7
  • GPU model and memory: RTX3070 - 8G
• How to use onnxruntime with flask

  Created a server that can run a session with multi-threads using Flask.

  When run 3 threads that the GPU's memory less than 8G, the program can run. But when run 4 threads that the GPU's memory will be greater than 8G, the program have error: onnxruntime::CudaCall CUBLAS failure 3: CUBLAS_STATUS_ALLOC_FAILED.

  I know that the problem is leaky of GPU's memory. But I hope that the program don't run crash. So I try to limit the number of threads, and set intra_op_num_threads = 2 or inter_op_num_threads = 2 or os.environ["OMP_NUM_THREADS"] = "2", but don't work.
  Try to 'gpu_mem_limit', don't work either

  import onnxruntime as rt
  from flask import Flask, request
  app = Flask(__name__)
  
  sess = rt.InferenceSession(model_XXX, providers=['CUDAExecutionProvider'])
  
  @app.route('/algorithm', methods=['POST'])
  def parser():
      prediction = sess.run(...)
  
  if __name__ == '__main__':
      app.run(host='127.0.0.1', port='12345', threaded=True)
  

  My understanding is that the Flask HTTP server maybe use different sess for each call.
  How can make each call use the same session of onnxruntime?

  System information

  • OS Platform and Distribution: Windows10
  • ONNX Runtime version: 1.8
  • Python version: python 3.7
  • GPU model and memory: RTX3070 - 8G
• pth to onnx model convertion

  I am trying to convert the pth format to onnx, while converting, I face this issue,

  RuntimeError: Exporting the operator _convolution_mode to ONNX opset version 13 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

  pytorch version is 1.11.0+cpu

  model -

      cnn = nn.Sequential()
      cnn.add_module('l1', nn.Conv2d(3, 32, 2, 1, "same"))
      cnn.add_module('l2', nn.ReLU())
      cnn.add_module('l3', nn.MaxPool2d(3, 3))
  
      ...
  
      cnn.add_module('fl', nn.LogSoftmax())
  
      torch.onnx.export(model, torch.zeros(16, 3, 224, 224).to(device), save_file, verbose=True, opset_version=13,
                  export_params=True,
                  training=torch.onnx.TrainingMode.EVAL,
                  do_constant_folding=True,
                  input_names = ['cnn.l1'], output_names=['cnn.fl'],
                            dynamic_axes={'cnn.l1': {0: 'batch'},
                                          'cnn.fl': {0: 'batch'}
                                          })
  

  In this, I got two warnings,

  UserWarning: Using padding='same' with even kernel lengths and odd dilation may require a zero-padded copy of the input be created (Triggered internally at ../aten/src/ATen/native/Convolution.cpp:744.)

  Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.

  How to successfully convert the pth file to onnx, thanks