scrape data from web page using JS only

2020-11-25 06:28 Learning imported from Stackoverflow

I want to scrape Player list from this website and want to show same in another website. i want to target whole div with id acceptance-lists__details-section and show same in another website. i was looking at scrapping example but most of them are JS + Node. is it possible to scrap only with JS any pointer would be helpful.

See also questions close to this topic

Creating a linked list object using js
I want to make a linked list using custom Object that pushes a value, pop a value, display all its content, remove an item from a specific place, and insert at a specific place as long as the value is missing from the sequence otherwise through an exception. All of the properties should be defined using data descriptor, prevent them from being deleted, iterated, or being modified.

I can do no more than this ... I'm new to js.
```
        var linkedList = {};

       /* linkedList.name = 'Ahmed';
        [].push.call(linkedList, 'sad', "sd");
*/
        Object.defineProperty(linkedList, "name", {
            value: "mohamed",
            writable: false,
            configurable: false,
            enumerable: false
        })
        linkedList.next = {'sd':'as'};
```
Any help? thanks in advance

Trying to trigger a CSS animation with a button press through Javascript

I'm trying to trigger a CSS animation with a button press by using Javascript. I've used other question and answers here, but to no avail. My code seems like it should work -- what am I missing? When I click the button, the background color of the div which I specify should change color over 2 seconds to light blue. I've tried changing the color of both my Body and my Test div, but nothing changes. My alert() triggers, which is confusing as it is in the same function as my colorChange.

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>&lt;model-viewer&gt; example</title>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />

    <script>
      function colorChange() {
        alert("The button works");
        document.getElementByID('test').style.animation="changeColor";
      }
    </script>

    <style>
      body {
        background-color: darkblue;
        -webkit-transition: all 2s ease;
        -moz-transition: all 2s ease;
        -o-transition: all 2s ease;
        -ms-transition: all 2s ease;
        transition: all 2s ease;
      }
      
      .test {
        width: 500px;
        height: 500px;
        background-color: pink;
      }

      @keyframes changeColor {
        to {
          background-color: lightblue;
        }
      }
    </style>
  </head>

  <body id="body">
    <div class="test"></div>
    
    <button id="button" onclick="colorChange()">test animation</button>
  </body>
</html>

No output from (GET) Ajax Result From Php Array with json_encode

I have a GET form that gets a Php Array and json encodes it. The Issue I am having is the success data is not displaying. I want the success to display data in a Alert or console but for some reason it's not working. I have tried many options. Thanks for your help.

PS: I know the GET and all files work because when I got the script the Ajax response result would populate/append a table successfully. I am modifying it.

Here is the php index file with the AJAX:

<!doctype html>
<html>
<head>
    <title>Return JSON response from AJAX using jQuery and PHP</title>
    <link href="style.css" type="text/css" rel="stylesheet">
    <script src="jquery-3.1.1.min.js" type="text/javascript"></script>
</head>
<body>
        
<script>        
 $(document).ready(function() {
    $.ajax({
        url: 'ajaxfile.php',
        type: 'get',
        dataType: 'JSON',
        success: function(data) {
            var obj = jQuery.parseJSON(data);
            alert(obj.username);
        }
    });
});
</script>  
</body>
</html>

Here is the php file that queries the database and encodes/array json:

<?php

$return_arr = array();

$sql = "SELECT * FROM users  ORDER BY NAME";
$result = $conn->query($sql);


// Check database connection first
if ($conn->query($sql) === FALSE) {
    echo 'database connection failed';
    die();
} else {
    while($row = $result->fetch_array()) {
        $id = $row['id'];
        $username = $row['username'];
        $name = $row['name'];
        $email = $row['email'];
  
        $return_arr[] = array("id" => $id,
                    "username" => $username,
                    "name" => $name,
                    "email" => $email);
    }
    
    // Encoding array in JSON format
    echo json_encode($return_arr);
}
?>

echo json_encode array output from the php file looks like below (test content):

[{"id":"1","username":"jiten","name":"Jiten singh\t","email":"jiten9"},{"id":"2","username":"kuldeep","name":"Kuldeep","email":"kuldee"},{"id":"3","username":"mayank","name":"Mayank","email":"mayank"},{"id":"9","username":"mohit","name":"Mohit","email":"mohit"},{"id":"13","username":"mukesh","name":"Mukesh","email":"mukesh"},{"id":"6","username":"nitin","name":"Nitin","email":"niti"},{"id":"12","username":"palash","name":"Palash","email":"palash"},{"id":"7","username":"rahul","name":"Rahul singh","email":"rahul"},{"id":"10","username":"rohit","name":"Rohit singh","email":"rohit"},{"id":"8","username":"shreya","name":"Shreya","email":"shreyam"},{"id":"11","username":"sonarika","name":"Sonarika","email":"sonarika"},{"id":"5","username":"vijay","name":"Vijay","email":"vijayec"},{"id":"14","username":"vishal","name":"Vishal Sahu\t","email":"visha"},{"id":"4","username":"yssyogesh","name":"Yogesh singh","email":"yoges"}]

when I make the success result (data) and alert(data), this is what I get,empty objects

How with Material UI to align input fields and buttons in the correct way in a dialog

In my React app, I have to align correctly the items in a dialog of MUI.

As per the screenshot, I have an issue to align the input fields Date and time. They should be aligned in the center and the Date input should start aligned with the title text above and the Time input should align with the button on the far right Save and close.

The second main issue is the buttons, I need to position Cancel and Save and close on the right and on the same row but on the left side, the button Remove call.

I have an issue making these alignments.

The code of the dialog

const useStyles = makeStyles(theme => ({
  root: {
    margin: 0,
    padding: theme.spacing(1),
    '& .MuiFilledInput-root': {
      borderRadius: 0,
    },
  },
  dialogTitle: {
    marginTop: theme.spacing(2),
  },
  container: {
    margin: theme.spacing(1),
    width: '80%',
  },
  textField: {
    marginLeft: theme.spacing(1),
    marginRight: theme.spacing(1),
  },
  button: {
    marginRight: theme.spacing(1),
    height: 40,
  },
  paper: {
    overflowY: 'unset',
  },
  closeButton: {
    position: 'absolute',
    left: '93%',
    top: '3%',
    color: 'gray',
  },
  buttonRight: {
    justifyContent: 'flex-end',
  },
}));

 <Dialog
      open={open}
      fullWidth
      maxWidth="md"
      classes={{ paper: classes.paper }}
    >
      <DialogTitle className={classes.dialogTitle}>
        <Typography variant="h1" gutterBottom>
          Schedule a call
        </Typography>
        <InfoCallMessage call={appointment} />
      </DialogTitle>
      {!loading ? (
        <>
          <DialogContent className={classes.root}>
            <Grid
              className={classes.container}
              container
              justify="flex-end"
              alignItems="center"
              spacing={1}
            >
              <Grid item xs={8}>
                <MuiPickersUtilsProvider utils={LuxonUtils}>
                  <DatePicker
                    className={classes.input}
                    disableToolbar
                    variant="inline"
                    label="Date"
                    format="cccc, LLLL dd"
                    helperText=""
                    value={date}
                    margin="normal"
                    onChange={newDate => {
                      handleDateOnChange({ newDate });
                    }}
                    inputVariant="filled"
                    fullWidth
                    minDate={new Date()}
                  />
                </MuiPickersUtilsProvider>
              </Grid>

              <Grid item xs={4}>
                <TextField
                  key={time}
                  id="time"
                  label="Time"
                  type="time"
                  defaultValue={time}
                  className={classes.textField}
                  InputLabelProps={{
                    shrink: true,
                  }}
                  variant="filled"
                  margin="normal"
                  onChange={e => {
                    const { value } = e.target;
                    setTime(value);
                  }}
                  fullWidth
                />
              </Grid>
              <ErrorDateTimeIsAfter />
            </Grid>
          </DialogContent>
          <DialogActions className={classes.buttonRight}>
            <IconButton
              className={classes.closeButton}
              onClick={() => closeDialog()}
            >
              <CloseIcon />
            </IconButton>
            <Grid justify="space-between" container spacing={1}>
              <Grid item>
                <Button
                  color="secondary"
                  variant="contained"
                  onClick={() => closeDialog()}
                  className={classes.button}
                >
                  Remove call
                </Button>
              </Grid>
              <Grid item>
                <Button
                  color="primary"
                  variant="outlined"
                  onClick={() => closeDialog()}
                  className={classes.button}
                >
                  Cancel
                </Button>
              </Grid>
              <Grid item>
                <Button
                  color="primary"
                  variant="contained"
                  onClick={() => handelSave()}
                  className={classes.button}
                  disabled={!isAfter(parseISO(`${date}T${time}`), new Date())}
                >
                  Save and Close
                </Button>
              </Grid>
            </Grid>
          </DialogActions>
        </>
      ) : (
        <Loading />
      )}
    </Dialog>

Digitally signed code for the web verified by browser

I wish to serve a web application where only digitally signed code from a trusted source is allowed to execute for a given domain.

To me it seems like such an obvious step for browsers to work towards implementing this.

Yes it is probably very messy technically, perhaps bordering impossible. But the upside would be enormous in my view.

Anyone knows why this hasn't been done?
How do fb ads projects collect data?

this is more a general question, but does anyone know, how websites like http://dropispy.com/ collect fb ads data and allow you to query FB ads data in time (week/month/year ago)?

An obvious answer would be - with FB Ads api, but with it, you have to query FB pages and can not query back in time.
How to scroll up and down and interact with web elements in a dynamic web page using Selenium Webdriver?
automation experts!

For quite a while I have been experiencing a specific issue related to web elements interaction within a web page. Precisely, I am not able to capture specific web page elements from a data table. None of Copy web element functions actually work - I have tried with Xpath, full Xpath, tag name, class name, id or partial (link) text after Right-click--> Inspecting the target page. Sadly, I am experiencing the following error once I execute the script:

raise exception_class(message, screen, stacktrace) selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified

It would seem that even though I try with Selenium webdriver or BeautifulSoup4/requests, none of the methods work. With BS4 I attempt the following:
```
resp = requests.get("https://business.google.com/local/business/")
soup = BeautifulSoup(resp.text, 'html.parser')
l = soup.find("ol", {"class": "olName"})

for el in l.find_all("li"):
    print(el.text)
```
In addition to the ol class there are multiple nested li lines that represent a table row from within the web page. Each value aria-label value behind it must be extracted.

Would be looking forward to any suggestions on how to proceed from now on, I seem to be banging my head against the wall here...
Error in (function (..., row.names = NULL, check.rows = FALSE, check.names = TRUE, : arguments imply differing number of rows: 17, 3
I am trying to convert a table from a website into a table I could read in R.

url <- "https://www.pro-football-reference.com/teams/cle/2020.htm#all_games"
```
pfr_raw <- url %>%
     read_html() %>%
     html_table() %>%
     as.data.frame()
```
But when I do this it throws this error -> `Error in (function (..., row.names = NULL, check.rows = FALSE, check.names = TRUE, : arguments imply differing number of rows: 17, 3'

The thing is if I use the link but 2019 instead of 2020 it works. How do I fix this issue? Thank You

Related questions

scrape data from web page using JS only

See also questions close to this topic