Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie
Hi all,
Vanilla are planning an update to the site on April 24th (next Wednesday). It is a major PHP8 update which is expected to boost performance across the site. The site will be down from 7pm and it is expected to take about an hour to complete. We appreciate your patience during the update.
Thanks all.

Python: repeating data, how to parse for easy access?

Options
  • 15-04-2013 1:19pm
    #1
    Tar.Aldarion
    Moderators, Science, Health & Environment Moderators, Social & Fun Moderators, Society & Culture Moderators Posts: 60,082 Mod ✭✭✭✭


    I have data stored in a variable:
    proc = subprocess.Popen(cmd_rancli, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
proc_stdout  = proc.communicate(ran_opt_get_access_data)[0]
print proc_stdout

    which is for the most part repeating, the nnumber iterations can vary. Now all this data isn't very intuitive but I know what each number stands for, eg FREQ OFFSET = Value(int): 9 and so on.

    My variable holds this (and a lot more repeating in the same fashion
    Request successful
       Protocol:  BSMIS Rx
    MessageType:  81 (GetAttributesResponse)
 ObjectInstance:
            RDN:
        ClassId:    1014 (AP)
     InstanceId:    0
  TransactionId:  66
  AttributeList:
    AttributeId:   4192
       Sequence:
       Sequence:
     Value(int):     1
  Value(string):     2013-02-26T15:01:11Z
       Sequence:
       Sequence:
     Value(int):       10564
     Value(int):       13
     Value(int):       388
     Value(int):       0
     Value(int):       -321
     Value(int):       83
  Value(string):       272
  Value(string):       05
     Value(int):       67
     Value(int):       67
     Value(int):       708
     Value(int):       896
     Value(int):       31
     Value(int):       128
     Value(int):       -12
     Value(int):       -109
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10564
     Value(int):       13
     Value(int):       108
     Value(int):       0
     Value(int):       -11
     Value(int):       83
  Value(string):       272
  Value(string):       05
     Value(int):       67
     Value(int):       67
     Value(int):       708
     Value(int):       1796
     Value(int):       31
     Value(int):       128
     Value(int):       -12
     Value(int):       -109
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10589
     Value(int):       16
     Value(int):       388
     Value(int):       0
     Value(int):       -15
     Value(int):       79
  Value(string):       272
  Value(string):       05
     Value(int):       67
     Value(int):       67
     Value(int):       708
     Value(int):       8680
     Value(int):       31
     Value(int):       125
     Value(int):       -16
     Value(int):       -110
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10589
     Value(int):       15
     Value(int):       108
     Value(int):       0
     Value(int):       -16
     Value(int):       81
  Value(string):       272
  Value(string):       05
     Value(int):       67
     Value(int):       67
     Value(int):       708
     Value(int):       8105
     Value(int):       31
     Value(int):       126
     Value(int):       -14
     Value(int):       -109
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10637
     Value(int):       40
     Value(int):       233
      Value(int):       0
     Value(int):       -11
     Value(int):       89
  Value(string):       272
  Value(string):       03
     Value(int):       30052
     Value(int):       1
     Value(int):       5
     Value(int):       54013
     Value(int):       33
     Value(int):       103
     Value(int):       -6
     Value(int):       -76
     Value(int):       1
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10662
     Value(int):       46
     Value(int):       234
     Value(int):       0
     Value(int):       -15
     Value(int):       85
  Value(string):       272
  Value(string):       03
     Value(int):       30052
     Value(int):       1
     Value(int):       5
     Value(int):       54016
     Value(int):       33
     Value(int):       97
     Value(int):       -10
     Value(int):       -74
     Value(int):       1
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10712
     Value(int):       51
     Value(int):       12
     Value(int):       0
     Value(int):       -24
     Value(int):       91
  Value(string):       272
  Value(string):       01
     Value(int):       4013
     Value(int):       254
     Value(int):       200
     Value(int):       2973
     Value(int):       3
     Value(int):       62
 Value(int):       -4
     Value(int):       -63
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10737
     Value(int):       15
     Value(int):       224
     Value(int):       0
     Value(int):       -16
     Value(int):       82
  Value(string):       272
  Value(string):       01
     Value(int):       3020
     Value(int):       21
     Value(int):       21
     Value(int):       40770
     Value(int):       33
     Value(int):       128
     Value(int):       -13
     Value(int):       -108
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10762
     Value(int):       14
     Value(int):       450
     Value(int):       0
     Value(int):       -7
     Value(int):       78
  Value(string):       272
  Value(string):       01
     Value(int):       3020
     Value(int):       21
     Value(int):       21
     Value(int):       53215
     Value(int):       29
     Value(int):       125
     Value(int):       -17
     Value(int):       -113
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10762
     Value(int):       15
 Value(int):       224
     Value(int):       0
     Value(int):       -7
     Value(int):       85
  Value(string):       272
  Value(string):       01
     Value(int):       3020
     Value(int):       21
     Value(int):       21
     Value(int):       50770
     Value(int):       33
     Value(int):       128
     Value(int):       -10
     Value(int):       -105
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10762
     Value(int):       14
     Value(int):       124
     Value(int):       0
     Value(int):       -7
     Value(int):       78
  Value(string):       272
  Value(string):       01
     Value(int):       3020
     Value(int):       10
     Value(int):       10
     Value(int):       56880
     Value(int):       32
     Value(int):       128
     Value(int):       -17
     Value(int):       -113
     Value(int):       0
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10812
     Value(int):       11
     Value(int):       135
     Value(int):       0
     Value(int):       -14
     Value(int):       81
  Value(string):       272
  Value(string):       02
     Value(int):       36002
     Value(int):       1
     Value(int):       11
     Value(int):       43159
     Value(int):       31
 Value(int):       130
     Value(int):       -14
     Value(int):       -113
     Value(int):       1
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       10837
     Value(int):       42
     Value(int):       23
     Value(int):       0
     Value(int):       -9
     Value(int):       89
  Value(string):       272
  Value(string):       02
     Value(int):       36002
     Value(int):       1
     Value(int):       11
     Value(int):       53529
     Value(int):       31
     Value(int):       99
     Value(int):       -6
     Value(int):       -74
     Value(int):       1
     Value(int):       -20
     Value(int):       -111
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
     Value(int):     54
       Sequence:
     Value(int):     13
  Value(string):     2013-02-26T15:02:09Z
       Sequence:
       Sequence:
     Value(int):       2
     Value(int):       12
     Value(int):       7
     Value(int):       0
     Value(int):       9
     Value(int):       70
  Value(string):       272
  Value(string):       02
     Value(int):       20003
     Value(int):       0
     Value(int):       0
     Value(int):       15535
     Value(int):       0
     Value(int):       0
     Value(int):       0
     Value(int):       0
     Value(int):       1
     Value(int):       100
     Value(int):       100
Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       5
     Value(int):       15
     Value(int):       44
     Value(int):       0
     Value(int):       -205
     Value(int):       77
  Value(string):       272
  Value(string):       02
     Value(int):       20003
     Value(int):       0
     Value(int):       0
     Value(int):       15632
     Value(int):       0
     Value(int):       0
     Value(int):       0
     Value(int):       0
     Value(int):       1
     Value(int):       100
     Value(int):       100
     Value(int):       -1
     Value(int):       -1
     Value(int):       0
       Sequence:
     Value(int):       7
     Value(int):       25
     Value(int):       9
     Value(int):       0
     Value(int):       0
     Value(int):       84
  Value(string):       272
  Value(string):       02
     Value(int):       20002
     Value(int):       0
     Value(int):       0
     Value(int):       50883
     Value(int):       0
     Value(int):       0
     Value(int):       0
     Value(int):       0
     Value(int):       1
     Value(int):       100
     Value(int):       100
     Value(int):       -1
     Value(int):       -1
     Value(int):       0  

//etc

    However something like this this occurs every now and again, not in every iteration:
    Value(int):     13
  Value(string):     2013-04-03T08:23:26Z
       Sequence:
       Sequence:

    So these repeat-ish, what would be an easy way to parse this data so that I could look at say the first element of each repeating series and so on to compare them? Can I make a struct? Do I need a dictionary, or how do I do this? If i made a dictionary, If the line is "Value(int): 10" I can just do int(line.split(":")[1])) and stick that in at the index. I assume I would need to split up each iteration into its own variable and assign each a dictionary of their own or something? I wouldn't know how many dictionaries there was and so on, and it doesn't seem much easier to compare data.

    Anyway, how would you do it?


Comments

  • Options
    #2
    padraig_f
    Registered Users Posts: 3,287 ✭✭✭padraig_f


    Not really sure what the data means, so that may inform the decision, but looking at it, I don't think a dictionary is suitable. Dictionaries require unique keys, and I don't see a natural key to the data.

    If I was storing it, I'd use a class for the header stuff, and a list for the int/string data (as in python, the same list can include both strings and ints). e.g.
    class ProcessData(object):

   def __init__(self):
      self.Protocol = None
      self.MessageType = 0
      ...
      self.values = [] # add ints/strings here


  • Options
    #3
    Tar.Aldarion
    Moderators, Science, Health & Environment Moderators, Social & Fun Moderators, Society & Culture Moderators Posts: 60,082 Mod ✭✭✭✭Tar.Aldarion


    Yeah maybe it isn't suitable. Perhaps I could split the data at "sequence:" into different lists, and then split them iwth \n and see if there is 22 elements in each list, then use the data. As each repeat should have 22 elements.


  • Options
    #4
    padraig_f
    Registered Users Posts: 3,287 ✭✭✭padraig_f


    Tar.Aldarion wrote: »
    Yeah maybe it isn't suitable. Perhaps I could split the data at "sequence:" into different lists, and then split them iwth \n and see if there is 22 elements in each list, then use the data. As each repeat should have 22 elements.

    Ok, I didn't get that there was patterns to the data. Then yeah, maybe a list of lists. e.g.
    class ProcessData(object):

   def __init__(self):
      self.Protocol = None
      self.MessageType = 0
      ...
      self.lists = []

# Parsing individual sequence...

def parseSequence(self):
   newlist = []
   # add values to newlist
   ...
   # at end of sequence, add list to self.lists
   self.lists.append(newlist)

# processing the list of lists...
def processLists(self):

   for list in self.lists:
      if len(list) == 22:
         # process 22-type list
      elif len(list) == 2:
         # process 2-type list


Advertisement