Difference between tf.train.shuffle_batch_join and tf.train.shuffle_batch

See also questions close to this topic

• Python Kafka Streaming API - Binning

  I am using python kafka stream binning example given in this, Python Kafka Streaming API

  I am able to generate the data using generator.py file given under winton-kafka-streams/examples/binning/, whereas when i run the binning.py file from the same folder, i got the below issue. Could someone help me, to resolve this?

  

  
• Change color of missing values in Seaborn heatmap

  Consider the example of missing values in the Seaborn documentation:

  corr = np.corrcoef(np.random.randn(10, 200))
  mask = np.zeros_like(corr)
  mask[np.triu_indices_from(mask)] = True
  sns.heatmap(corr, mask=mask, vmax=.3, square=True)
  

  

  How do I change the color of the missing values to, for example, black? The color of the missing values should be specified independent of the color scheme of the heatmap, it may not be present in the color scheme.

  I tried adding facecolor = 'black' but that didn't work. The color can be affected by e.g. sns.axes_style("white") but it isn't clear to me how that can be used to set an arbitrary color.
• Xpath + Scrapy + Python : data point couldn't be scraped

  This is the XML structure:

  <tr>
     <td>
         <font size="3">
             <strong>Location:</strong> 
             Hiranandani Gardens, Powai
         </font>
     </td> 
  </tr>  
  

  I want to extract : Hiranandani Gardens, Powai

  I tried with these:

  response.xpath('//font[@size="3"]/strong[text()="Location:"]/text()').extract()
  
  response.xpath('//font[@size="3"]/strong[text()="Location:"]').extract()
  

  Both returned an empty list.

  Note: we must have to use the text of tag, i.e., "Location:". Otherwise, there are many other places on the site where the same XML structure is used. So, it'll fetch many more unnecessary things apart from the desired value if the text of strong tag is not used.
• Date Quarters in 'Series' object python

  I'm trying to count the value in each date in order to count witch quarter is high frequency I try these methods not working so far... please help :) thanks. Note: stime is 'Series' object

  stime=df['timestamp']
  #print (df['timestamp'].filter(like='08', axis=0)
  #stime.filter(like='2018-07-18')
  #stime.between_date('2018-01-01','2018-02-01', include_start=True, include_end=True)
  #stime.month
  
• Python to sort tag with different attributes

  I have a list from links = set(soup.findAll('a')), now I want to sort them, but links = sorted(links) got error, what can I sort it?

  The original list and any else attributes:

  <a href="//weathernews.jp/typhoon/">台風</a>
  <a class="list" href="https://weathernews.jp/s/topics/201809/190035/?fm=tp_index"></a>
  <a class="list" href="https://weathernews.jp/s/topics/201809/130155/?fm=tp_index"></a>
  <a href="//weathernews.jp/warning/">警報・注意報</a>
  <a class="list" href="https://weathernews.jp/s/topics/201809/140125/?fm=tp_index"></a>
  <a class="list" href="https://weathernews.jp/s/topics/201809/170145/?fm=tp_index"></a>
  

  I want to sort it to

  <a class="list" href="https://weathernews.jp/s/topics/201809/190035/?fm=tp_index"></a>
  <a class="list" href="https://weathernews.jp/s/topics/201809/170145/?fm=tp_index"></a>
  <a class="list" href="https://weathernews.jp/s/topics/201809/140125/?fm=tp_index"></a>
  <a class="list" href="https://weathernews.jp/s/topics/201809/130155/?fm=tp_index"></a>
  <a href="//weathernews.jp/typhoon/">台風</a>
  <a href="//weathernews.jp/warning/">警報・注意報</a>
  
• Changing the encoding of a .csv in Python 3

  I have a .csv file whose encoding is cp1252 and I want to transform it into utf8. To the new .csv I want to apply a function and analyze the data. My attempt was to iterate through the rows and get a list of cells:

  def convertEncoding(path):
  with open(path, 'r', encoding = 'cp1252') as file:
      csv_2 = csv.reader(file, delimiter='\t', quotechar='"')
      for row in csv_2:
          f = ', '.join(row)
          print(f)
  

  How do I create and use the .csv document with the right encoding in a big function that processes the entire file (and it even uses the name of the file)?

  def function(myfile):
      string_array = myfile.split('_')
      data_start_string = string_array[1].split('-')
      data_end_string = string_array[3].split('-')
      ...
  
• Keras InvalidArgumentError: Incompatible shapes: [1,8,32] vs. [1,10,32]

  It stops training at the beginning of the first epoch.

  The error is:

  InvalidArgumentError: Incompatible shapes: [1,8,32] vs. [1,10,32]
       [[Node: training_8/RMSprop/gradients/loss_8/time_distributed_9_loss/mul_grad/BroadcastGradientArgs = BroadcastGradientArgs[T=DT_INT32, _class=["loc:@train.../Reshape_1"], _device="/job:localhost/replica:0/task:0/device:CPU:0"](training_8/RMSprop/gradients/loss_8/time_distributed_9_loss/mul_grad/Shape, training_8/RMSprop/gradients/loss_8/time_distributed_9_loss/mul_grad/Shape_1)]]
  

  I'm not sure if the error might be the format of the training data. This is a sample of train_x:

  [array([ 0,  1,  2,  3,  4,  5,  6,  3,  7,  8,  9, 10, 11, 12, 13, 14], dtype=int32), array([15, 16, 17, 18, 19], dtype=int32), array([20, 16, 17, 18, 21, 22, 23, 24, 25, 26], dtype=int32), array([27,  1, 17, 28, 18, 29, 30, 31, 24, 32], dtype=int32), array([33,  1, 17,  3, 34, 35, 36, 37, 18, 38], dtype=int32), array([39, 16, 40, 28, 41, 42], dtype=int32), array([39,  1, 40, 28, 41, 43], dtype=int32), array([44,  6,  3, 45], dtype=int32), array([15, 16, 40, 46, 47, 48,  3, 49, 50, 51, 52, 53], dtype=int32), array([ 0, 54, 28, 55, 56, 57, 58, 59], dtype=int32)]
  

  This is a sample of train_label

  [array([0, 1, 2, 1, 2, 3, 0, 1, 4, 2, 2, 4, 5, 6, 0, 7], dtype=int32), array([0, 5, 8, 9, 9], dtype=int32), array([10,  5,  8,  9,  7,  9,  7,  7], dtype=int32), array([10,  1,  8,  1,  9,  9,  3,  0,  7,  2,  7], dtype=int32), array([10,  1,  8,  1,  9,  9,  9,  7,  9,  7], dtype=int32), array([0, 5, 8, 1, 2, 2], dtype=int32), array([0, 1, 8, 1, 2, 2], dtype=int32), array([11,  0,  1,  9], dtype=int32), array([ 0,  5,  8,  7, 12, 13,  1,  2,  9,  9,  5,  6], dtype=int32), array([ 0, 14,  1,  2,  2, 12,  4, 15], dtype=int32)]
  

  This is the code:

  n_vocab = len(unique_words) #1517
  model = Sequential()
  model.add(Embedding(n_vocab,100))
  model.add(Convolution1D(128, 5, border_mode='same', activation='relu'))
  model.add(Dropout(0.25))
  model.add(GRU(100,return_sequences=True))
  model.add(TimeDistributed(Dense(n_classes, activation='softmax')))
  model.compile('rmsprop', 'categorical_crossentropy')
  
  n_epochs = 30
  train_x = [np.asarray(x, dtype=np.int32) for x in encoded_sentences]
  train_label = [np.asarray(x, dtype=np.int32) for x in encoded_POS]
  
  for i in range(n_epochs):
      print("Training epoch {}".format(i))
  
      bar = progressbar.ProgressBar(maxval=len(train_x))
      for n_batch, sent in bar(enumerate(train_x)):
          label = train_label[n_batch]
          # Make labels one hot
          label = np.eye(n_classes)[label][np.newaxis,:]
          # View each sentence as a batch
          sent = sent[np.newaxis,:]
  
          if sent.shape[1] > 1: #ignore 1 word sentences
              model.train_on_batch(sent, label)
  
• Is there a way to name a tensorflow variable based on the value of another tensor-variable

  I want to be able to do the following

  n = str(tf.constant(2))
  v = tf.get_variable(name=n, shape=(256,256), initializer=tf.contrib.layers.xavier_initializer())
  

  but doing this is converts n to a string representation of tensorflow variable i.e

  "<tf.Tensor 'Const_4:0' shape=() dtype=int32>"
  
• How To Let Tensorflow Use All CPU

  My Linux has 2 CPUs (16 cores per CPU). Each core has 2 threads.

  CPU usage rate only 50% less if i use tf.Session(tf.ConfigProto())

  CPU usage rate only 40% less if i use tf.Session(tf.ConfigProto(device_count={"CPU": 64}))

  Cpu usage rate is 30-70% if i use tf.ConfigProto(device_count={"CPU": 32}, inter_op_parallelism_threads=1, intra_op_parallelism_threads=64, )

  And in this configuration all cores are working, but the usage rate is only 30-80%

  I have tested in Windows Server (1 cpu / 4 cores). Cpu usage rate is more than 90%.

  what should I code can let Tensorflow use all cpu?
• MNIST data denormalising does not give me back the same

  This is part of my learning. I Understood the normalization is really helping to improve accuracy, and hence divided by 255 the mnist values. This will divide all the pixels by 255, and hence all the pixels of 28*28 will have the values in range from 0.0 to 1.0 .

  Now i tired to multiply the same with 255,this essentially means we should get the original value back. but when i display the picture, both the original and de-normalised pictures are different.

  (trainX, trainY), (testX, testY) = mnist.load_data()
  
  
  plt.subplot(2,2,1)
  plt.imshow(trainX[143])
  
  trainX /= 255
  
  plt.subplot(2,2,2)
  plt.imshow(trainX[143])
  
  
  trainX *= 255
  
  plt.subplot(2,2,3)
  plt.imshow(trainX[143])
  plt.show()
  

  OUtput:

  

  What am i missing?. Anything related to float and int data type of the input data?
• Which is best way of logging to file in C?

  I'm dealing with some deep learning model written in C.
  I wanna make some log file to check it later.
  log will be a line for each step.
  Each step will take few seconds.

  Sometimes I use keyboard interrupt to stop procedure

  Ways I thought are:

  // Way 1
  fp = fopen("log.txt", "a");
  for each step:
    fprintf(fp, "Log content\n");  
  flose(fp);
  

  I think way 1 may have lower file open/close overhead.
  But when I use keyboard interrupt to stop procedure, log file will be never closed properly.
  Is it OK?
  Or can pass file pointer as argument of the my own signal handler?

  // Way 2
  for each step:
    fp = fopen("log.txt", "a");
    fprintf(fp, "Log content\n");  
    fclose(fp);
  

  I think way 2 will have file open/close overhead for each step, could this slow down whole performance?
  Could it be crucial?

  // Way 3
  for each step:
    fprintf(buffer, "### Log content ###");  
  
    if step % 100 == 0:
      fp = fopen("log.txt", "a");
      fprintf(fp, buffer);  
      fclose(fp);
      flush buffer;
  

  for way 3, i consider 2 ways of buffer:
  1. array of string
  2. one long string with line feed between items

  Which way works best?
  It depends?

  Or if there is a good library for logging, could you recommend it to me?
• Using Scikit-Learn's pipelines to combine a transformers and estimator

  I try to use Scikit-Learn's Pipeline function to organize our transformers and estimator, and having problem with building a pipeline that combines one_hot_transformer with a LinearRegression() estimator. It is challenging to connect the following ones

  from sklearn.preprocessing import OneHotEncoder
  cat_feats = np.array([[1,10],[2,20],[3,10],[4,20],[3,10],[2,20],[1,10]])
  OneHotEncoder(sparse=False).fit_transform(cat_feats)
  one_hot_transformer = OneHotEncoder(sparse=False).fit_transform(X,y)
  
  from sklearn.pipeline import Pipeline
  linear_est = Pipeline([one_hot_transformer], LinearRegression())
  linear_est.fit(X,y)
  predicted = linear_est.predict(X)
  grader.score('intro_ml__linear_model', linear_est.predict)
  
• MongoDB retrieve an aggregation match with a distinct value

  This is my query $ [

     { 
         "$match" : {
             "$and" : [
                 {
                     "timestamp" : {
                         "$gte" : ISODate("2018-09-10T00:00:00.000+0000"), 
                        "$lte" : ISODate("2018-09-15T00:00:00.000+0000")
                     }
                 }, 
                 {
                     "meta.event" : "x"
                 }, 
                 {
                     "session.username" : "y"
                 }
             ]
         }
     }, 
     { 
         "$project" : {
             "user" : "$session.username", 
             "sessionId" : "$session.id", 
             "date" : "$timestamp"
         }
     }
  

  ]

  I need to edit this query to retrieve documents without get duplicated sessionId but a distinct value.
• process_item pipeline not called

  I'm new to python and scrapy

  After scraping process I tried to save database to mysqlite, Follow by this src : https://github.com/sunshineatnoon/Scrapy-Amazon-Sqlite( from url)

  My problem is database was created successfully but items can't be inserted to database because process_item not called

  EDIT

  I paste the source code from github link above setting.py

  ITEM_PIPELINES = {
      'amazon.pipelines.AmazonPipeline': 300
      }
  

  pipelines.py

  # -*- coding: utf-8 -*-
  
  # Define your item pipelines here
  #
  # Don't forget to add your pipeline to the ITEM_PIPELINES setting
  # See: http://doc.scrapy.org/en/latest/topics/item-pipeline.html
  import sqlite3
  import os
  con = None
  
  
  class AmazonPipeline(object):
  
      def __init__(self):
          self.setupDBCon()
          self.createTables()
  
      def process_item(self, item, spider):
          print('---------------process item----')
          self.storeInDb(item)
          return item
  
      def setupDBCon(self):
          self.con = sqlite3.connect(os.getcwd() + '/test.db')
          self.cur = self.con.cursor()
  
      def createTables(self):
          self.dropAmazonTable()
          self.createAmazonTable()
  
      def dropAmazonTable(self):
          #drop amazon table if it exists
          self.cur.execute("DROP TABLE IF EXISTS Amazon")
  
      def closeDB(self):
          self.con.close()
  
      def __del__(self):
          self.closeDB()
  
      def createAmazonTable(self):
          self.cur.execute("CREATE TABLE IF NOT EXISTS Amazon(id INTEGER PRIMARY KEY NOT NULL, \
              name TEXT, \
              path TEXT, \
              source TEXT \
              )")
  
          self.cur.execute("INSERT INTO Amazon(name, path, source ) VALUES( 'Name1', 'Path1', 'Source1')")
          print ('------------------------')
          self.con.commit()      
  
  
      def storeInDb(self,item):
          # self.cur.execute("INSERT INTO Amazon(\
          #     name, \
          #     path, \
          #     source \
          #     ) \
          # VALUES( ?, ?, ?)", \
          # ( \
          #     item.get('Name',''),
          #     item.get('Path',''),
          #     item.get('Source','')
          # ))
  
          self.cur.execute("INSERT INTO Amazon(name, path, source ) VALUES( 'Name1', 'Path1', 'Source1')")
  
          print ('------------------------')
          print ('Data Stored in Database')
          print ('------------------------')
          self.con.commit()  
  

  spiders/amazonspider.py

  import scrapy
  import urllib
  from amazon.items import AmazonItem
  import os
  
  
  class amazonSpider(scrapy.Spider):
      imgcount = 1
      name = "amazon"
      allowed_domains = ["amazon.com"]
      '''
      start_urls = ["http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=backpack",
                    "http://www.amazon.com/s/ref=sr_pg_2?rh=i%3Aaps%2Ck%3Abackpack&page=2&keywords=backpack&ie=UTF8&qid=1442907452&spIA=B00YCRMZXW,B010HWLMMA"
                    ]
      '''
      def start_requests(self):
          yield scrapy.Request("http://www.amazon.com/s/ref=sr_ex_n_3?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&bbn=10445813011&ie=UTF8&qid=1442910853&ajr=0",self.parse)
  
          for i in range(2,3):
              yield scrapy.Request("http://www.amazon.com/s/ref=lp_360832011_pg_2?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&page="+str(i)+"&bbn=10445813011&ie=UTF8&qid=1442910987",self.parse)
  
  
  
      def parse(self,response):
          #namelist = response.xpath('//a[@class="a-link-normal s-access-detail-page  a-text-normal"]/@title').extract()
          #htmllist = response.xpath('//a[@class="a-link-normal s-access-detail-page  a-text-normal"]/@href').extract()
          #imglist = response.xpath('//a[@class="a-link-normal a-text-normal"]/img/@src').extract()
          namelist = response.xpath('//a[@class="a-link-normal s-access-detail-page s-overflow-ellipsis a-text-normal"]/@title').extract()
          htmllist = response.xpath('//a[@class="a-link-normal s-access-detail-page s-overflow-ellipsis a-text-normal"]/@href').extract()
          imglist = response.xpath('//img[@class="s-access-image cfMarker"]/@src').extract()
          listlength = len(namelist)
  
          pwd = os.getcwd()+'/'
  
          if not os.path.isdir(pwd+'crawlImages/'):
              os.mkdir(pwd+'crawlImages/')
  
          for i in range(0,listlength):
              item = AmazonItem()
              item['Name'] = namelist[i]
              item['Source'] = htmllist[i]
  
              urllib.urlretrieve(imglist[i],pwd+"crawlImages/"+str(amazonSpider.imgcount)+".jpg")
              item['Path'] = pwd+"crawlImages/"+str(amazonSpider.imgcount)+".jpg"
              amazonSpider.imgcount = amazonSpider.imgcount + 1
              yield item
  

  Result after run scrapy crawl amazone I have test.db created but item haven't inserted (I've checked my sqlite db.test), that mean process_item was not run

  build result

  2018-09-18 16:38:38 [scrapy.utils.log] INFO: Scrapy 1.5.1 started (bot: amazon)
  2018-09-18 16:38:38 [scrapy.utils.log] INFO: Versions: lxml 4.2.1.0, libxml2 2.9.8, cssselect 1.0.3, parsel 1.5.0, w3lib 1.19.0, Twisted 17.5.0, Python 3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37) - [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)], pyOpenSSL 18.0.0 (OpenSSL 1.0.2o  27 Mar 2018), cryptography 2.2.2, Platform Darwin-17.7.0-x86_64-i386-64bit
  2018-09-18 16:38:38 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'amazon', 'NEWSPIDER_MODULE': 'amazon.spiders', 'SPIDER_MODULES': ['amazon.spiders']}
  2018-09-18 16:38:38 [scrapy.middleware] INFO: Enabled extensions:
  ['scrapy.extensions.corestats.CoreStats',
   'scrapy.extensions.telnet.TelnetConsole',
   'scrapy.extensions.memusage.MemoryUsage',
   'scrapy.extensions.logstats.LogStats']
  2018-09-18 16:38:38 [scrapy.middleware] INFO: Enabled downloader middlewares:
  ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
   'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
   'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
   'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
   'scrapy.downloadermiddlewares.retry.RetryMiddleware',
   'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
   'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
   'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
   'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
   'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
   'scrapy.downloadermiddlewares.stats.DownloaderStats']
  2018-09-18 16:38:38 [scrapy.middleware] INFO: Enabled spider middlewares:
  ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
   'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
   'scrapy.spidermiddlewares.referer.RefererMiddleware',
   'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
   'scrapy.spidermiddlewares.depth.DepthMiddleware']
  ------------------------
  2018-09-18 16:38:38 [scrapy.middleware] INFO: Enabled item pipelines:
  ['amazon.pipelines.AmazonPipeline']
  2018-09-18 16:38:38 [scrapy.core.engine] INFO: Spider opened
  2018-09-18 16:38:38 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  2018-09-18 16:38:38 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023
  2018-09-18 16:38:38 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.amazon.com/s/ref=lp_360832011_pg_2?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&page=2&bbn=10445813011&ie=UTF8&qid=1442910987> from <GET http://www.amazon.com/s/ref=lp_360832011_pg_2?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&page=2&bbn=10445813011&ie=UTF8&qid=1442910987>
  2018-09-18 16:38:38 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.amazon.com/s/ref=sr_ex_n_3?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&bbn=10445813011&ie=UTF8&qid=1442910853&ajr=0> from <GET http://www.amazon.com/s/ref=sr_ex_n_3?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&bbn=10445813011&ie=UTF8&qid=1442910853&ajr=0>
  2018-09-18 16:38:39 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.amazon.com/backpacks/b?ie=UTF8&node=360832011> from <GET https://www.amazon.com/s/ref=sr_ex_n_3?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&bbn=10445813011&ie=UTF8&qid=1442910853&ajr=0>
  2018-09-18 16:38:39 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (301) to <GET https://www.amazon.com/Backpacks-Luggage-Travel-Gear/s?ie=UTF8&page=2&rh=n%3A360832011> from <GET https://www.amazon.com/s/ref=lp_360832011_pg_2?rh=n%3A7141123011%2Cn%3A10445813011%2Cn%3A9479199011%2Cn%3A360832011&page=2&bbn=10445813011&ie=UTF8&qid=1442910987>
  2018-09-18 16:38:40 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.amazon.com/Backpacks-Luggage-Travel-Gear/s?ie=UTF8&page=2&rh=n%3A360832011> (referer: None)
  2018-09-18 16:38:40 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.amazon.com/backpacks/b?ie=UTF8&node=360832011> (referer: None)
  2018-09-18 16:38:41 [scrapy.core.engine] INFO: Closing spider (finished)
  2018-09-18 16:38:41 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
  {'downloader/request_bytes': 1909,
   'downloader/request_count': 6,
   'downloader/request_method_count/GET': 6,
   'downloader/response_bytes': 140740,
   'downloader/response_count': 6,
   'downloader/response_status_count/200': 2,
   'downloader/response_status_count/301': 4,
   'finish_reason': 'finished',
   'finish_time': datetime.datetime(2018, 9, 18, 9, 38, 41, 53948),
   'log_count/DEBUG': 7,
   'log_count/INFO': 7,
   'memusage/max': 52600832,
   'memusage/startup': 52600832,
   'response_received_count': 2,
   'scheduler/dequeued': 6,
   'scheduler/dequeued/memory': 6,
   'scheduler/enqueued': 6,
   'scheduler/enqueued/memory': 6,
   'start_time': datetime.datetime(2018, 9, 18, 9, 38, 38, 677280)}
  2018-09-18 16:38:41 [scrapy.core.engine] INFO: Spider closed (finished)
  

  I've searching around but have no luck

  Thanks