仅使用 MySQL 查询删除重复项?

2021-11-20 00:00:00 sql mysql

我有一个包含以下列的表格:

I have a table with the following columns:

URL_ID    
URL_ADDR    
URL_Time

我想使用 MySQL 查询删除 URL_ADDR 列上的重复项.

I want to remove duplicates on the URL_ADDR column using a MySQL query.

不使用任何编程就可以做这样的事情吗?

Is it possible to do such a thing without using any programming?

推荐答案

考虑以下测试用例:

CREATE TABLE mytb (url_id int, url_addr varchar(100));

INSERT INTO mytb VALUES (1, 'www.google.com');
INSERT INTO mytb VALUES (2, 'www.microsoft.com');
INSERT INTO mytb VALUES (3, 'www.apple.com');
INSERT INTO mytb VALUES (4, 'www.google.com');
INSERT INTO mytb VALUES (5, 'www.cnn.com');
INSERT INTO mytb VALUES (6, 'www.apple.com');

我们的测试表现在包含的位置:

Where our test table now contains:

SELECT * FROM mytb;
+--------+-------------------+
| url_id | url_addr          |
+--------+-------------------+
|      1 | www.google.com    |
|      2 | www.microsoft.com |
|      3 | www.apple.com     |
|      4 | www.google.com    |
|      5 | www.cnn.com       |
|      6 | www.apple.com     |
+--------+-------------------+
5 rows in set (0.00 sec)

那么我们就可以使用多表DELETE语法如下:

Then we can use the multiple-table DELETE syntax as follows:

DELETE t2
FROM   mytb t1
JOIN   mytb t2 ON (t2.url_addr = t1.url_addr AND t2.url_id > t1.url_id);

... 这将删除重复的条目，只留下基于 url_id 的第一个 url:

... which will delete duplicate entries, leaving only the first url based on url_id:

SELECT * FROM mytb;
+--------+-------------------+
| url_id | url_addr          |
+--------+-------------------+
|      1 | www.google.com    |
|      2 | www.microsoft.com |
|      3 | www.apple.com     |
|      5 | www.cnn.com       |
+--------+-------------------+
3 rows in set (0.00 sec)

<小时>

更新 - 对上述新评论的进一步补充:

UPDATE - Further to new comments above:

如果重复的 URL 格式不同，您可能需要应用 REPLACE() 函数删除 www. 或 http:// 部分.例如:

If the duplicate URLs will not have the same format, you may want to apply the REPLACE() function to remove www. or http:// parts. For example:

DELETE t2
FROM   mytb t1
JOIN   mytb t2 ON (REPLACE(t2.url_addr, 'www.', '') = 
                   REPLACE(t1.url_addr, 'www.', '') AND 
                   t2.url_id > t1.url_id);

相关文章