based on one website , which contains statistical information, i have implemented basic web scrapping code and here it is :
import re
import requests
from bs4 import BeautifulSoup
content =requests.get("https://www.geostat.ge/ka/modules/categories/26/samomkhmareblo-fasebis-indeksi-inflatsia")
content =BeautifulSoup(content.content,'html.parser')
# print(content.prettify())
information =[]
for row in content.select('tbody tr'):
for data in row.find_all('td'):
if len(data.text.strip())!=0:
information.append(data.text.strip())
print(information)
it returns following information :
['2012', '2013', '2014', '2015', '2016', '2017', '2018', '2019', '2020', '2021', '2022', '2023', 'საშუალო წლიური წინა წლის საშუალო წლიურთან', '99.1', '99.5', '103.1', '104.0', '102.1', '106.0', '102.6', '104.9', '105.2', '109.6', '111.9', '102.5', 'დეკემბერი წინა წლის დეკემბერთან', '98.6', '102.4', '102.0', '104.9', '101.8', '106.7', '101.5', '107.0', '102.4', '113.9', '109.8', '100.4'
now first part before the text containing 'საშუალო' is year, rest of them are inflations between two text, so i have implemented this very manual code :
years =[]
average_annual =[]
december =[]
first_index =information.index('საშუალო წლიური წინა წლის საშუალო წლიურთან')
second_index =information.index('დეკემბერი წინა წლის დეკემბერთან')
for i in range(0,first_index):
years.append(int(information[i]))
print(years)
for i in range(first_index+1,second_index):
average_annual.append(float(information[i]))
print(average_annual)
for i in range(second_index+1,len(information)):
december.append(float(information[i]))
print(december)
it shows correct seperation :
[2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023]
[99.1, 99.5, 103.1, 104.0, 102.1, 106.0, 102.6, 104.9, 105.2, 109.6, 111.9, 102.5]
[98.6, 102.4, 102.0, 104.9, 101.8, 106.7, 101.5, 107.0, 102.4, 113.9, 109.8, 100.4]
my question is : is there more optimal way of doing this?thanks in advance