Is there a more efficient way to check number of columns?

2018-07-11 23:30 Wade imported from Stackoverflow

So I am trying to do some client side validation on an app I am working on, and I want to be able to check the number of columns in a CSV that the user uploads. The CSVs could potentially be very large (up to 30 thousand rows) so I'm worried that my current method, which reads the entire file, is not the best way to do it.

This is what I have right now.

// Function to validate the File Upload
$(document).ready(function() {
    $("#csv").change(function () {
        console.log("Read CSV Function Triggered");
        let theFile = document.getElementById("csv").files[0];
        let reader = new FileReader();
        reader.addEventListener('load', readFile);
        reader.readAsText(theFile);

        function readFile(event) {
            let csvText = event.target.result;
            let firstLine = csvText.split('\n').shift(); // First Line
            $("#result").html(firstLine);
        }
    });
});

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>

<input type="file" id="csv" />

<br>

<span id="result">0</span>

In the example I just make the first line display on the page, but in the app I have a parser library that I couldn't include in this example which converts the first line to an array and then I get the length of the array. So like I said before, this works, but I am worried that when large CSVs are uploaded it may affect browser performance.

Any suggestions?

See also questions close to this topic

Adding a constructor to an injected Project Class

I'm trying to add a constructor, which I call ctor, to Project, so I can add to it some properties when it's instantiated.

Here's the (absolutely ghastly) hack I'm using on Item to make this tick:

const canvas = document.querySelector('canvas')
paper.setup(canvas)

paper.Item.inject({
  ctor: function(args) {
    console.log('Item constructed!')
  }
})

// All Items are supposed to call `_initialize` so we "hook" there.
const Item_initialize = paper.Item.prototype._initialize
const Item_ctor = paper.Item.prototype.ctor

paper.Item.prototype._initialize = function(...args) {
  const initializer = Item_initialize.apply(this, args)
  if (Item_ctor) Item_ctor.apply(this)

  return initializer
}

const path = new paper.Path.Line({
	strokeColor: 'black',
  strokeWidth: 5,
  from: 100,
  to: 50
})

<script src="https://cdnjs.cloudflare.com/ajax/libs/paper.js/0.12.0/paper-core.js"></script>
<canvas></canvas>

Unfortunately the above trick doesn't work on Project:

const canvas = document.querySelector('canvas')
paper.setup(canvas)

paper.Project.inject({
  ctor: function(args) {
    // Not called
    console.log('Project constructed!')
  }
})

const Project_initialize = paper.Project.prototype._initialize
const Project_ctor = paper.Project.prototype.ctor

paper.Project.prototype._initialize = function(...args) {
  const initializer = Project_initialize.apply(this, args)
  if (Project_ctor) Project_ctor.apply(this)

  return initializer
}

const project = new paper.Project(canvas)

<script src="https://cdnjs.cloudflare.com/ajax/libs/paper.js/0.12.0/paper-core.js"></script>
<canvas></canvas>

What am I missing here?

How to delete div and all element inside div? Javascript, html, css

I have a function that clears the entire div and it disappears but still appears in the inspect (html). This is a real problem because we have this input type field on the email and I got this empty data in email. I only want when this value is not chosen to completely remove me from html and inspect. Look at my code and try to catch the error. The most important things in the whole code that you need to pay attention are onchange="checkSelected()" in html and first script tag which manipulate with that. It should simply become a display none but it still stands there.

<div class="modal fade" id="montageModal" tabindex="-1" role="dialog">
   <div class="modal-dialog" role="document">
      <div class="modal-content" style="display: flex;">



         <div class="modal-body">


            <form id="schedule_form" class="clearfix" action="index.php?route=api/reifenmontage" method="post">

               <div class="container-fluid">

                  <div class="step_1"  >
                     <h3 class="modal-title">Reifenmontage Termin buchen  </h3>
                     <div class="row termin_row">
                        <div class="col-xs-12 col-sm-4">
                           <div class="row">
                              <label>Pneu-Typ</label>
                           </div>
                        </div>
                        <div class="col-xs-12 col-sm-6">
                           <div class="row">
                              <select onchange="checkSelected()" class="form-control" name="pneu" id="pneu">
                                 <option value="Motorrad">Motorrad</option>
                                 <option value="Auto">Auto</option>
                              </select>
                           </div>
                        </div>
                     </div>
                     <div id="additionalRow"  class="row termin_row" >
                     <div id="reifenmontage-input" class="row termin_row">
                        <div class="col-xs-12 col-sm-4">
                           <div class="row">
                              <label>Mark und model</label>
                           </div>
                        </div>
                        <div class="col-xs-12 col-sm-4">
                           <div class="row">
                              <select name="marka" class="form-control" id="button-getdata">
                              </select>
                           </div>
                        </div>
                        <div class="col-xs-12 col-sm-4">
                           <div class="row">
                              <select name="model" class="form-control" id="result" > 
                              </select> 
                           </div>
                        </div>

                     </div>
                     </div>
                     <div class="row termin_row">
                        <div class="col-sm-4 col-xs-12">
                           <div class="row"><label>2. Terminvorschlag</label></div>
                        </div>
                        <div class="col-sm-4 col-xs-6">
                           <div class="row">
                              <div class="input-group date" id="date-2" data-termin="1">
                                 <input type='text' class="form-control" name="date[1]" />
                                 <span class="input-group-addon">um</span>
                              </div>
                           </div>
                        </div>
                        <div class="col-sm-4 col-xs-6">
                           <div class="row">
                              <div class="input-group time" id="time-2" data-termin="1">
                                 <input type='text' class="form-control" name="time[1]" />
                                 <span class="input-group-addon">Uhr</span>
                              </div>
                           </div>
                        </div>
                     </div>

                     <div class="row">
                        <div class="checkbox">
                           <label>
                              <input type="checkbox" name="accept" id="accept"> Ich akzeptiere die <a href="teilnahmebedingungen" target="_blank">Teilnahmebedingungen</a>
                           </label>
                        </div>
                     </div>

                     <div class="row text-center">
                        <button type="button" class="btn btn-primary btn-lg btn-submit" id="next_step" disabled="disabled">Anfrage senden</button>
                     </div>
                  </div>

                  <div class="step_2">
                     <h3 class="modal-title">Your contact info</h3>

                     <div class="">
                        <div class="form-group clearfix">
                           <input type="text" name="name" value="<?= $user['name'] ?>" placeholder="Name and Lastname" class="form-control name text" required />
                        </div>
                        <div class="form-group clearfix">
                           <input type="text" name="phone" value="<?= $user['phone'] ?>" placeholder="Phone" class="form-control phone text" required />
                        </div>
                        <div class="form-group clearfix">
                           <input type="email" name="email" value="<?= $user['email'] ?>" placeholder="Email" class="form-control email text" required />
                        </div>
                        <div class="text-center">
                           <button type="submit" id="submit" class="btn btn-default btn-lg btn-submit" >Suchen</button>
                        </div>
                     </div>

                  </div>

               </div>

            </form>

         </div>
         <div class="modal-footer">
            <button type="button" class="close" data-dismiss="modal" aria-label="Close">SCHLIESSEN</button>
         </div>
      </div>
   </div>
</div>

and my script tag

<script type="text/javascript">
   let selectItem = document.getElementById('pneu');
   let additionalRow = document.getElementById('additionalRow');
   function checkSelected() {
      if (selectItem.selectedIndex == "1") {
         additionalRow.style.display = 'none';
      } else {
         additionalRow.style.display = 'block';
      }
   }
</script> 

<script type="text/javascript">
$('#button-getdata').on('change', function() {
    $.ajax({
        url: 'index.php?route=api/reifenmontage/get_marka_data',
        type: 'post',
        data: $('#reifenmontage-input select'),
        dataType: 'json',
        beforeSend: function() {

        },
        success: function(json) {

            if (json['success']) {
               $("#result").empty();
                for (i in json['success']) {
                var element = json['success'][i];
                var o = new Option(element['model'], element['model']);
               $("#result").append(o);
                    html = "\t<option   value=\""+ element['model'] + "\">" + element['model'] + "</option>\n";
                    $("#result").append(o); 
                }
               //  document.getElementById("schedule_form").reset();
            }   

        }
    });
});
</script>  

<script type="text/javascript">

$.ajax({
 url: 'index.php?route=api/reifenmontage/mark',
 context: document.body,
 success: function(data) {
   const selectControl = $('#button-getdata');
   selectControl.html(data.map(ExtractData).join(''));
 }
});

function ExtractData(item) {
return ` <option value="${item.value}">${item.label}</option>`;
}

</script>

How to change the color of words in jumbotron using the reference $jumbotron?
I am learning web design and js .On my webpage, I define the default color is black in jumbotron h2.
```
.jumbotron h2{
    text-align: center;
    color:black;
}
```
define reference var $jumbotron = $('.jumbotron h2'); while running, I use $jumbotron.html('hellow world'); ...... to display some words and I want to change its color but I do not know how to set new color while running, how to get the color value and assign a new value for my variable $jumbotron? I have tried $jumbotron.color = or $jumbotron.color('') but they are wrong. I have not found any useful answers with google.

Match three game

I'm working on a project and I need to make a Match-Three game and I need to check with jQuery if 3 or more candies are next to each other but can't figure it out.

There's one column of 7 and this is the code to fill it with random images:

$(".element").attr('src', function() {
  var candy = Math.floor(Math.random() * 4 + 1);
  return "image/" + candy + ".png";
});

<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<div class="col-1">
  <img src="" class="element">
  <img src="" class="element">
  <img src="" class="element">
  <img src="" class="element">
  <img src="" class="element">
  <img src="" class="element">
  <img src="" class="element">
</div>

As you can see, I have 4 different candies. The thing that I need is for the program to check if 3 or more candies are aligned and remove them.

Any help or tip would be much appreciated.

Appending rows from one CSV to another in Python

I have looked at many solutions for this but cannot find one that works for what I want to do.

Basically I have 2 CSV files:

all.csv

1   Wed Oct 03  41.51093923 41.51093923 41.51093923 41.51093923         
2   Wed Oct 04                          
3   Wed Oct 05  41.43764015 41.43764015 41.43764015             
4   Wed Oct 06  41.21395681 41.21395681 41.21395681             
5   Wed Oct 07  42.07607442 42.07607442 42.07607442             
6   Wed Oct 08  42.0074109  42.0074109  42.0074109              
7   Wed Oct 09  41.21395681 41.21395681                 
8   Wed Oct 10  41.43764015 41.43764015 41.43764015 41.43764015 
9   Wed Oct 11  41.21395681 41.21395681 41.21395681 41.21395681

original.csv

10  Wed Oct 12  41.43764015             
11  Wed Oct 13                  
12  Wed Oct 14  42.07607442 42.07607442 42.07607442     
13  Wed Oct 15  41.43764015 41.43764015 41.43764015 41.43764015
14  Wed Oct 16  41.21395681 41.21395681 41.21395681 41.21395681
15  Wed Oct 17                  
16  Wed Oct 18  42.07607442 42.07607442 42.07607442

I want to append original.csv to all.csv, by simply taking all rows in original.csv and merging them underneath the last line in all.csv to get:

1   Wed Oct 03  41.51093923 41.51093923 41.51093923 41.51093923         
2   Wed Oct 04                          
3   Wed Oct 05  41.43764015 41.43764015 41.43764015             
4   Wed Oct 06  41.21395681 41.21395681 41.21395681             
5   Wed Oct 07  42.07607442 42.07607442 42.07607442             
6   Wed Oct 08  42.0074109  42.0074109  42.0074109              
7   Wed Oct 09  41.21395681 41.21395681                 
8   Wed Oct 10  41.43764015 41.43764015 41.43764015 41.43764015 
9   Wed Oct 11  41.21395681 41.21395681 41.21395681 41.21395681
10  Wed Oct 12  41.43764015             
11  Wed Oct 13                  
12  Wed Oct 14  42.07607442 42.07607442 42.07607442     
13  Wed Oct 15  41.43764015 41.43764015 41.43764015 41.43764015
14  Wed Oct 16  41.21395681 41.21395681 41.21395681 41.21395681
15  Wed Oct 17                  
16  Wed Oct 18  42.07607442 42.07607442 42.07607442

As you can see there is no headers for the data and the rows vary in length. This is just an example of the type of files I am working with but I want to get a solution that can work on any CSV.

I am working with Python3 and so far have tried using the pandas library but had no luck.

Any suggestions would be great, thanks.

How do I make this into a function for input files?

Short explanation:

I need to gather input files based on the file name without using expand(), I'm trying to make a custom function based on the code below but it's not working.

Long explanation:

I have the following python code that closely resembles what I need to do, but that needs some fixing and adapting to snakemake:

import glob
import pandas as pd
import os

samples = pd.read_csv('samples.tsv', sep='\t', dtype=str).set_index(["flowcell", "sample", "lane"], drop=False)
samples.index = samples.index.set_levels([i.astype(str) for i in samples.index.levels])  # enforce str in index

filename = pd.Series(samples['sample']).unique()
rgs = sorted(glob.glob(os.path.join('Outputs/MergeBamAlignment/%s*.bam' % filename)))
print(rgs)

I use this tsv file:

flowcell    sample  library lane    R1  R2
FlowCellX   SAMPLE1 libZ    L001    fastq/Sample1.R1.fastq.gz   fastq/Sample1.R2.fastq.gz
FlowCellX   SAMPLE1 libZ    L002    fastq/Sample1.R1.fastq.gz   fastq/Sample1.R2.fastq.gz
FlowCellX   SAMPLE1 libY    L003    fastq/Sample1.R1.fastq.gz   fastq/Sample1.R2.fastq.gz
FlowCellY   SAMPLE2 libY    L003    fastq/Sample2.R1.fastq.gz   fastq/Sample2.R2.fastq.gz
FlowCellX   SAMPLE2 libY    L003    fastq/Sample2.R1.fastq.gz   fastq/Sample2.R2.fastq.gz

And when I run the code I get this output:

['Outputs/MergeBamAlignment/SAMPLE1_L001_FlowCellX.merged.bam', 'Outputs/MergeBamAlignment/SAMPLE1_L002_FlowCellX.merged.bam', 'Outputs/MergeBamAlignment/SAMPLE1_L003_FlowCellX.merged.bam', 'Outputs/MergeBamAlignment/SAMPLE2_L003_FlowCellX.merged.bam', 'Outputs/MergeBamAlignment/SAMPLE2_L003_FlowCellY.merged.bam']

I think this is wrong though, it should (probably) look like this:

['Outputs/MergeBamAlignment/SAMPLE1_L001_FlowCellX.merged.bam', 'Outputs/MergeBamAlignment/SAMPLE1_L002_FlowCellX.merged.bam', 'Outputs/MergeBamAlignment/SAMPLE1_L003_FlowCellX.merged.bam']
['Outputs/MergeBamAlignment/SAMPLE2_L003_FlowCellX.merged.bam', 'Outputs/MergeBamAlignment/SAMPLE2_L003_FlowCellY.merged.bam']

I can produce the correct output with this for loop:

for s in pd.Series(samples['sample']).unique():
    filename = glob.glob(os.path.join('Outputs/MergeBamAlignment/%s*.bam' % s))
    print filename

But I understand for loops aren't allowed in python functions so that's out of the question.

So how can I produce lists (?) with the input files from each sample without using the expand() function but instead using a custom function? The reason expand() is out of the question is because the answer to my previous question broke the previous rule since {{sample}} seems to be messing up zip in the previous rule's expand() function.

Any suggestion would be greatly appreciated.

optimize adding column to CSV file (~300GB) with python
I want to add a column to CSV file, wich is the difference of two other columns of the same file, I use python (pandas) to do that and this is what i do :
```
import pandas as pd
row = ['times1','times2']
for df1 in pd.read_csv('C:/SET/parti_no_diff.CSV',skipinitialspace=True, usecols=row, chunksize=10**7):
    df1['time_difference'] = (df1['times2'].astype('datetime64[s]')-df1['times1'].astype('datetime64[s]')).abs()
    df1.to_csv('E:/SET/parti_with_diff_seconds.csv',mode='a')
```
I use a machine with 12GB RAM, and external hard disk of 2TB (5200RPM) (the input are not on the same hard disk as output), the program take more than 24h, how can I optimize it?
How to validate a non required filed with yup
I'm using yup with formik for validations. I want to validate a non-required field with yup. Here test is a non-required field but when user enters we are blocking him to enter some special characters. when the user didn't enter anything then also it is validating( i don't need this validation until user enters something). Can anyone help me, Thanks in advance
```
   const validationSchema = function (values) {
    return Yup.object().shape({
       test: Yup.string()
      .matches(/^[^<&>`#]+$/,'test should not contain `, &, #, <, > \n')
    })
  }
```
How can a W3C type HTML Validator be made in C Language?

I'm trying to make a validator to catch all the possible errors that can occur in a sample HTML document. I'm trying to write this code in C Language. How do I make sure that it works just like HTML W3C Validator? And catches error? Can someone show a sample code snippet?
customize validation functions in Laravel
I am curious about ($value & ($value - 1)) != 0 that how it is working in below validation to know number is power of 2?!
```
function ($attribute, $value, $fail) {
                    if ($value == 0 || ($value & ($value - 1)) != 0) {
                        $fail($attribute . ' is not power of 2!');
                    }
                }
```
If I would like get numbers between numbers of power 2 in addition to power 2 numbers, what shall I do? can I use and modify this command? (e.g. numbers: 1,2,3,4,6,8,12,16,...)

Related questions

Is there a more efficient way to check number of columns?

See also questions close to this topic