Screener data to csv

At first I thought this should be a piece of cake... but how do I save the data I screened to a csv?
In general, I am somewhat puzzled how to properly use csvs to store data with the TWS API


- Below just a very general code on how I basically structured my scanner -

Code:

''' General file to automate screening'''
#%% Imports
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.utils import iswrapper
from ibapi.scanner import ScannerSubscription
from ibapi.tag_value import TagValue

from threading import Thread
import time

#%% Client and Wrapper
class StockScanner(EWrapper, EClient):
    ''' Serves as the client and the wrapper '''

    def __init__(self, addr, port, client_id):
        EClient. __init__(self, self)

        # Connect to TWS
        self.connect(addr, port, client_id)
        self.count = 0
       
        # Launch the client thread
        thread = Thread(target=self.run)
        thread.start()
   
    @iswrapper
    def scannerData(self, reqId, rank, details, distance, benchmark, projection, legsStr):
        # Print the symbols in the returned results
        print('{}: {}'.format(rank, details.contract.symbol))
        self.count += 1
       
    @iswrapper
    def scannerDataEnd(self, reqId):
        # Print the number of results
        print('Number of results: {}'.format(self.count))
   
    def error(self, reqId, code, msg):
        print('Error {}: {}'.format(code, msg))
#%% Main
def main():

    # Create the client and connect to TWS
    client = StockScanner('127.0.0.1', 7497, 0)
    time.sleep(0.5)

    # Create the ScannerSubscription object   
    ss = ScannerSubscription()
    ss.instrument = 'STK'
    ss.locationCode = 'STK.US'
    # Top % Gainers Since Open
    ss.scanCode = 'TOP_OPEN_PERC_GAIN'
    ss.abovePrice = 1.0
    ss.belowPrice = 1000.0

    # Set additional filter criteria
    tagvalues = []
    tagvalues.append(TagValue('changePercAbove', '0%'))

    # Request the scanner subscription
    client.reqScannerSubscription(0, ss, [], tagvalues)

    # Sleep while the request is processed
    time.sleep(0.5)     
    client.disconnect()
   
if __name__ == '__main__':
    main()

 

What does the
client.reqScannerSubscription
output look like?

 

In scannerData(): instead of using print(...) save or append the required data to a csv file.

 

https://stackoverflow.com/questions/61604175/save-data-from-tws-api-to-csv-file/62491076#62491076

Here they showed a potential solutions for Contract Details, however I am not too sure how to implement this for scanner Data as the function is automatically updating itself.

 

So I did it differently and am kinda confused about my output...
Can someone help me on how I correct this output?

I just want to have the number and then the symbol per "row"

Code:

Code:

    @iswrapper
    def scannerData(self, reqId, rank, details, distance, benchmark, projection, legsStr):
        # Print the symbols in the returned results
        self.count += 1
        data = [[pd.Series(rank), pd.Series(details.contract.symbol)]]
        self.df = self.df.append(data, ignore_index = True)

    @iswrapper
    def scannerDataEnd(self, reqId):
        # Print the number of results
        print(self.df)
        print('Number of results: {}'.format(self.count))

Output:

Code:

   rank symbol                     0                        1
0   NaN    NaN   0    0
dtype: int64   0    AAL
dtype: object
1   NaN    NaN   0    1
dtype: int64   0    XLF
dtype: object

[...]

Code:

48  NaN    NaN  0    48
dtype: int64   0    HST
dtype: object
49  NaN    NaN  0    49
dtype: int64   0    ERI
dtype: object
Number of results: 50