Ufdjrw

I just started using python for some web page scraping and BeautifulSoup seems to be recommended everywhere.

I have the content like below:

<table class="table with-row-highlight table-archive">
 <tbody>
 <tr>
 <td>
 <div class="user-tagline ">
 <span class="username " data-avatar="aaaaaaa">player1</span>
 <span class="user-rating">(1357)</span>
 <span class="country-flag-small flag-113" tip="Portugal"></span>
 </div>
 <div class="user-tagline ">
 <span class="username " data-avatar="bbbbbbb">player2</span>
 <span class="user-rating">(1387)</span>
 <span class="country-flag-small flag-70" tip="Indonesia"></span>
 </div>
 </td>
 <td>
 <a class="clickable-link text-middle" href="https://www.chess.com/live/game/2249663029?username=belemnarmada" target="_self">
 <div class="pull-left">
 <span class="game-result">1</span>
 <span class="game-result">0</span>
 </div>
 <div class="result">
 <i class="icon-square-minus loss" tip="Lost"></i>
 </div>
 </a>
 </td>
 <td class="text-center">
 <a class="clickable-link" href="https://www.chess.com/live/game/2249663029?username=belemnarmada" target="_self">30 min</a>
 </td>
 <td class="text-right">
 <a class="clickable-link text-middle moves" href="https://www.chess.com/live/game/2249663029?username=belemnarmada" target="_self">25</a>
 </td>
 <td class="text-right miniboard">
 <a class="clickable-link archive-date" href="https://www.chess.com/live/game/2249663029?username=belemnarmada" target="_self">Aug 9, 2017</a>
 </td>
 </tr>
 100 <tr></tr> here
 </tbody>
</table>

My code stops here, how do I write the python code to loop all the <tr></tr> pair and extract all the class for each <span> pair in each <td> pair?

edit

I think maybe I didn't explain clearly here, what your code returns are the name of class in that HTML while what I am looking for are the correspondent values, e.g. there is a class username, I want to get its value of player1 and player2; there is a class country-flag-small flag-70 I want to get tip=Indonesia

This should do the trick:

import requests
from bs4 import BeautifulSoup

res = requests.get('someLink')
soup = BeautifulSoup(res.text)


classes = []
for element in soup.find_all(class_=True):
 classes.extend(element["class"])
print(classes)

I tested this using your html file and got the following results:

['table', 'with-row-highlight', 'table-archive', 'user-tagline', 'username', 'user-rating', 'country-flag-small', 'flag-113', 'user-tagline', 'username', 'user-rating', 'country-flag-small','flag-70', 'clickable-link', 'text-middle', 'pull-left', 'game-result', 'game-result', 'result', 'icon-square-minus', 'loss', 'text-center', 'clickable-link', 'text-right', 'clickable-link', 'text-middle', 'moves', 'text-right', 'miniboard', 'clickable-link', 'archive-date']

Do note that you will have to pip3 install requests if you haven't already

Also, if you want to test this with a file on your computer, you can do this:

from bs4 import BeautifulSoup

file = open('/path/To/Your/HtmlFile.html', 'r')
lines = file.read()
soup = BeautifulSoup(lines)


classes = []
for element in soup.find_all(class_=True):
 classes.extend(element["class"])
print(classes)