Extracting data from Pinterest using gRPC, Python

I want to grab all chats of my Pinterest account
I have a Proto Service:

syntax = "proto3";


service Pinterest{
    rpc GetConversations (request_chat_info) returns (chat_response);
}



message request_chat_info{
    string conversation_id = 1;
    string csrftoken = 2;
    string _b = 3;
    string _pinterest_sess = 4;
}


message chat_response{
    string type = 1;
    string id = 2;
    string text = 3;
}


message chat_response_array{
    repeated chat_response messages = 1;
}

and this is my Pinterest Servicer:

# GRPC Service
class PinterestService(pb2_grpc.PinterestServicer):
    def GetConversations(self, request, context):
        conversation_id = request.conversation_id
        csrftoken = request.csrftoken
        _b = request._b
        _pinterest_sess = request._pinterest_sess
        chats = _exec(
            get_chat,
            {"conversation_id": conversation_id, "csrftoken": csrftoken, "_b": _b, "_pinterest_sess": _pinterest_sess}
        )
        
        return pb2.chat_response_array(messages=chats)

and main Program is something like this:

# ENDPOINTS
CHAT_API = "https://www.pinterest.com/resource/ConversationMessagesResource/get/"




# Execute Fucntion
def _exec(func, params):
    return func(**params)



# Make Requests here
def _get(url:str, cookies:Dict = None, headers:Dict = None) -> requests.Response:
    response = requests.request("GET", url=url, cookies=cookies, headers=headers)
    response.raise_for_status()
    return response


# Chat Parser Function
def _chat_parser(chat_dict: Dict) -> Dict:
    return {
        "type": chat_dict.get("type", ""),
        "id": chat_dict.get("id", ""),
        "text": chat_dict.get("text", ""),
    }


# Function to handle GRPC
def get_chat(conversation_id:str, csrftoken:str, _b:str, _pinterest_sess:str) -> Dict:
    options = {"page_size":25,"conversation_id":conversation_id,"no_fetch_context_on_resource":False}
    _cookies = {"csrftoken":csrftoken, "_b":_b, "_pinterest_sess":_pinterest_sess}
    query = {"data": json.dumps({"options":options})}
    encoded_query = urlencode(query).replace("+", "%20")
    url = "{}?{}".format(CHAT_API, encoded_query)
    msg_counter = 0
    while True:
        try:
            return _chat_parser(_get(url, _cookies).json()["resource_response"]["data"][msg_counter])
        except IndexError:
            break
        finally:
            msg_counter += 1

I need to get all CHAT and I Don’t know how to do that!
The Response JSON in Pinterest is exactly Like this:

["resource_response"]["data"][0]
["resource_response"]["data"][1]
["resource_response"]["data"][2]
["resource_response"]["data"][...]

Based on Messages Count the last number change from 0 to any number
I don’t know how to handle that!
does the fault in my proto or what?
should I use stream in proto, if yes, bidirectional stream or for Client, Server…
Thank you for Helping me.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

I found the answer
it must be server stream and use for loop and yield

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0