Fastest Way To Execute a Query From 15,000,000+ rows
My database is around 15-20 billion lines long. I am planning to make a breach checking service (similar to leakcheck). After putting a small part of the database on a MariaDB test database I had installed, I noticed it takes around 4 seconds to run a query from a small portion (18m) of my total amount (15-20b) of rows. The table is formatted as follows: breach_id email password username ip phone number
Everything is in 1 table. If anyone can help me get the query time to under 1s I would be so appreciative.
VPS CPU: AMD EPYC 7313 16-Core Processor Ram: 62G Mem, 1.5G Swap Storage: 5TB
I have no issue in changing the database I use. The service will be non-profit and aims to help people
response of show create table data_info;
+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| data_info | CREATE TABLE `data_info` (
`database_id` int(11) DEFAULT NULL,
`email` varchar(255) DEFAULT NULL,
`password` varchar(255) DEFAULT NULL,
`username` varchar(255) DEFAULT NULL,
`ip` varchar(255) DEFAULT NULL,
`phone_number` varchar(255) DEFAULT NULL,
FULLTEXT KEY `email` (`email`,`password`,`username`,`ip`,`phone_number`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-----------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.000 sec)
MariaDB [hackcheck]> explain select * from data_info where email = "fbi"
-> ;
+------+-------------+-----------+------+---------------+------+---------+------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-----------+------+---------------+------+---------+------+----------+-------------+
| 1 | SIMPLE | data_info | ALL | email | NULL | NULL | NULL | 18233866 | Using where |
+------+-------------+-----------+------+---------------+------+---------+------+----------+-------------+
How many English words
do you know?
do you know?
Test your English vocabulary size, and measure
how many words do you know
Online Test
how many words do you know
Powered by Examplum