c# - HtmlAgilityPack isn't getting all the html code/text from a web page -


for starters, thank in advance!

i able extract section of code web page looks similar following block of code.

<div id="playerstats">   <div id="hp"><span class="title">hp:</span></div>   <div id="mp"><span class="title">mp:</span></div>   <div id="magicresist"><span class="title">magic resist</span></div>   <div id="physicaldefend"><span class="title">physical defence</span></div>   <div id="phycriticalreducerate"><span class="title">strike resist</span></div>   <div id="phycriticaldamagereduce"><span class="title">strike fortitude</span></div>   <div id="physicalright"><span class="title">main hand attack</span></div>   <div id="accuracyright"><span class="title">main hand accuracy</span></div>   <div id="criticalright"><span class="title">main hand critical</span></div>   <div id="physicalleft"><span class="title">off hand attack</span></div>   <div id="accuracyleft"><span class="title">off hand accuracy</span></div>   <div id="criticalleft"><span class="title">off hand critical</span></div>   <div id="attackspeed"><span class="title">attack speed</span></div>   <div id="magicalboost"><span class="title">magic boost</span></div>   <div id="magicalaccuracy"><span class="title">magic accuracy</span></div>   <div id="magicalcriticalright"><span class="title">crit spell</span></div>   <div id="castingtimeratio"><span class="title">casting speed</span></div>   <div id="block"><span class="title">block</span></div>   <div id="dodge"><span class="title">evasion</span></div> </div> 

from following uri for character statistics page of video game. (and should see table of stats in middle of page.) if use browser's function similar google chrome's f-12 view html source code, notice there values in between /span , /div similar following code:

<div id="playerstats">   <div id="hp"><span class="title">hp:</span>"12213"</div>   <div id="mp"><span class="title">mp:</span>"4000"</div>   <div id="magicresist"><span class="title">magic resist</span>"4618"</div>   <div id="physicaldefend"><span class="title">physical defence</span>"1725"</div>   <div id="phycriticalreducerate"><span class="title">strike resist</span>"1518"</div>   <div id="phycriticaldamagereduce"><span class="title">strike fortitude</span>"392"</div>   <div id="physicalright"><span class="title">main hand attack</span>"201"</div>   <div id="accuracyright"><span class="title">main hand accuracy</span>"201"</div>   <div id="criticalright"><span class="title">main hand critical</span>"201"</div>   <div id="physicalleft"><span class="title">off hand attack</span>"201"</div>   <div id="accuracyleft"><span class="title">off hand accuracy</span>"201"</div>   <div id="criticalleft"><span class="title">off hand critical</span>"201"</div>   <div id="attackspeed"><span class="title">attack speed</span>"201"</div>   <div id="magicalboost"><span class="title">magic boost</span>"201"</div>   <div id="magicalaccuracy"><span class="title">magic accuracy</span>"201"</div>   <div id="magicalcriticalright"><span class="title">crit spell</span>"201"</div>   <div id="castingtimeratio"><span class="title">casting speed</span>"201"</div>   <div id="block"><span class="title">block</span>"201"</div>   <div id="dodge"><span class="title">evasion</span>"201"</div> </div> 

and go on, using following code retrieve first block of html code described above.

htmldocument doc = new htmldocument(); doc.load(mytestfile);  foreach(var node in doc.documentnode.selectnodes("//div[@id='playerstats']/div/span")) {     console.writeline(node.innertext + " " + (node.nextsibling != null ?  node.nextsibling.innertext : null)); } 

i have used webrequest, webclient, webbrowser , htmlweb-agilitypack classes pull html document down web. however, important part wish extract not being pulled down in document values associated hp, mp, etc... expected values described in second block of html code above.

how can code bring down simple text in document me parse well?

player info loaded dynamically calling http://psykopats.net/loadaion.php post method , few parameters, 1 of player , identifies player. in case, parameters were:

server:66 type:1 player:299345 

you can take @ this question see how use post webclient.

the response json string that, among other things, contains looking for:

stat: {basecriticalresist:0, magiccriticalresist:0, physicaldefend:1402, basemagicalspeed:1,…} accuracyleft: 2617 accuracyright: 2617 agi: 110 airresist: 0 attackspeed: 1.1 baseaccuracyleft: 1705 baseaccuracyright: 1705 baseagi: 110 baseairresist: 0 baseattackspeed: 1.1 baseblock: 837 basecastingtimeratio: "1.0" basecriticaldefend: 0 basecriticalleft: 53 basecriticalresist: 0 basecriticalright: 103 basedex: 110 basedodge: 1839 basedp: 4000 baseearthresist: 0 basefireresist: 0 basehealboost: 0 basehealskillboost: 0 basehp: 6688 basekno: 90 basemagcriticaldamagereduce: 0 basemagcriticalreducerate: 0 basemagiccriticaldefend: 0 basemagiccriticalresist: 0 basemagicresist: 1384 basemagicalaccuracy: "" basemagicalattack: 0 basemagicalboost: 0 basemagicalcriticalleft: 50 basemagicalcriticalright: 50 basemagicalspeed: 1 basemovespeed: 6 basemp: 4318 baseparry: 1847 basephycriticaldamagereduce: 0 basephycriticalreducerate: 190 basephysicaldefend: 1162 basephysicalleft: 255 basephysicalright: 234 basestr: 110 basevit: 100 basewaterresist: 0 basewill: "" block: 837 castingtimeratio: 0.98 criticaldefend: 0 criticalleft: 602 criticalresist: 0 criticalright: "" dex: 110 dodge: 2272 dp: 4000 earthresist: "" fireresist: 0 healboost: 0 healskillboost: 0 hp: 11210 kno: 90 magcriticaldamagereduce: 0 magcriticalreducerate: 38 magiccriticaldefend: 0 magiccriticalresist: 0 magicresist: 1725 magicalaccuracy: 1201 magicalattack: 0 magicalboost: 0 magicalcriticalleft: 50 magicalcriticalright: 50 magicalspeed: "1.0" movespeed: 7.56 mp: 4618 parry: "" phycriticaldamagereduce: 201 phycriticalreducerate: 392 physicaldefend: 1402 physicalleft: 658 physicalright: 658 str: 110 vit: 100 waterresist: 0 will: 0 

sample code:

system.net.webclient wc = new system.net.webclient(); byte[] data = wc.uploadvalues(     "http://psykopats.net/loadaion.php",     new system.collections.specialized.namevaluecollection(){         {"server", "66"},         {"type", "1"},         {"player", "299345"}}); string json = system.text.encoding.ascii.getstring(data); 

Comments