1、extract the Username from the Profile Page

2、get the tweets

3、get the list of follwers and followings of UserA

4、construct the social network between the targets

5、develop the webdemo

6、Prepare for the presentation

This assignment is based on the project:https://github.com/Altimis/Scweet. If you want to use the code , please pay attention to the MIT Liensence of the Altimis's project.Here are the steps:

Step1: download the source code from the link or download it as python library:

pip3 install Scweet==1.6

Ps: If you download it with command pip3, please pay attention to the function Log_in() in the file Utils.py.Due to the log_in page on twitter changed, Scweet 1.6 couldn't work as you wish. You'd better change the Utils.py with the same one of this:https://github.com/Altimis/Scweet

Step2: call the functions

With the code as an example, you could collect tweets within the limitation.

from Scweet.scweet import scrape

data = scrape(words=['bitcoin','ethereum'], since="2021-10-01", until="2021-10-05", from_account = None,interval=1, headless=False, display_type="Top", save_images=False, lang="en",resume=False, filter_replies=False, proximity=False)

ps: "data" above is pd.dataframe, so you can use some packages to save it as .csv or .json and more formats.

With the code as an example, you could collect followers and followings.(Two functions return the dictionary.)

from Scweet.user import get_user_information, get_users_following, get_users_followers
users = ['@username']
env_path = ".env"

following = get_users_following(users=users, env=env_path, verbose=0, headless=False, wait=2, limit=50, file_path=None)

followers = get_users_followers(users=users, env=env_path, verbose=0, headless=False, wait=2, limit=50, file_path=None)

ps:The file '.env' should include your twitter account imformation, required to log in with the Selenium. You need to edit it and save it in the root directory of the project.

# file_name = '.env'
SCWEET_USERNAME=your username
SCWEET_PASSWORD=your password

Go to the target's profile page to get the Username like "@Jerry" not "Jerry", with the file "List name of ...."

Before the starting the subtask, we have to know how to set three parameters 'from account','to account'and 'mention account' in the func:scrape()

Here are an example:

data = scrape(since="2021-12-05", until="2021-12-06", from_account = 'Bugeater12',to_account='SenatorFischer', interval=1,headless = False,display_type="Top", save_images=False, lang="en",resume=False, filter_replies=False, proximity=False)

you could get this:

Here are an example:

data = scrape(since="2021-12-04", until="2021-12-05", from_account = 'SenatorFischer',to_account='growneb', interval=1,headless = False,display_type="Top", save_images=False, lang="en",resume=False, filter_replies=False, proximity=False)

you could get this tweet finally.

Here are an example(here, the senator is the one that we don't care):

data = scrape(since="2021-11-17", until="2021-11-18", from_account = 'SenatorFischer',words =['ASA_Soybeans','UsChamber'], interval=1,headless = False,display_type="Top", save_images=False, lang="en",resume=False, filter_replies=False, proximity=False)

you could get this tweet finally.

Now, we discuss how to get the tweets where User1 and User2 appear together. Maybe, we can also divide it in 3 conditions:

(1)User1 tweets, and User2 is mentioned in User1's tweet;

(2)User1 tweets, and User2 reply to User1's tweet as a comment;

(3)Someone tweets, and User1 & User2 are mentioned is the tweet.

In our project, we get the list of Sens & Reps. Then traverse all Sens & Reps as User1 and User2 in above 3 conditions, recycling 544 * (544 - 1) times in each one.

Use the Selenium Framework to get the result.

Here, we just output three 2-dimension matrixes(544 * 544) as the result of constructing the social network. If there is a kind of relationship between Rep_i & Rep_j, then matrix_i_j = 1, otherwise 0.

There are still one intersting and useful task:

（'Opinion Leader' means some people who get the most influential in a social community）. However, among all Reps&Sens, there must be some different clusters of senators supporting their own opinion leader(s), which means it's possble that the root opnion leader doesn't exit. As a result of this, here are the steps for this task:

First, we have to make an assumption that: If senatorA follows senatorB, it means senatorA supports senatorB

(1)Get all Reps&Sens into different clusters

(2)Extract the common Followings of all Reps&Sens in each cluster as the Opinion Leaders of each cluster.

(3)If the number of the result is smaller than the threshold(we can set ahead), then recycle the step(1) - step(2).

Finally, we can get the reslut. What's more, here are an recommended model for step(1): get clusters based on calculating the similarity between any two elemnts, Reps & Sens in our project. This is a link of the paper supporting the model:https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6755451) Honestly speaking, we don't have to ,and we can't construct a complex model as the one in that papaer, just because we don't have so many kinds of relationships between Reps. So the model will be easier.