RESTful Bugs?

Topics: Developer Forum
Developer
Oct 4, 2013 at 6:24 PM
Hi all,

I have some issues with RESTful module.

1) REST may not be selecting correct controllers.
e.g. if we have two controllers say 'employee' and 'employeeCodes' then it may select 'employee' controller all the time. In this case the parameter count is also incorrectly determined to be one more than the actual count. This results in landing on a incorrect 'action'.

The problem code in handleRequest() is as follows..

*-------------+
FOR i = 1 TO THIS.Controllers.Count
cResource = THIS.Controllers.Keys[i]
cController = THIS.Controllers.Values[i]
IF ATC(cResource, cUri) > 0
j = ATC("/" + cResource, cUri)    &&ATC("/" + cResource + "/", cUri)
IF j > 0
 EXIT
ENDIF
ENDIF
ENDFOR
*-----------+


Question.

What was the reason to replace

ATC("/" + cResource + "/", cUri)

With

ATC("/" + cResource, cUri)


2) In the same function, it seems that the punishment to return a 'Boolean' value in the following eval() code is little extreme

cHTMLOut = EVALUATE("oController." + cAction + "()")

I found it impossible to trace this problem since no error is generated. I also found this problem in sample code of customers controller in two actions.

i.e.
getAction() (..when !found())
getcustomerrs() (return value)


3) Any error generated/detected within this function are passed back to the client side as error strings. This results in longer debugging time of custom actions, since I was trying to locate the problem in my calling JavaScript/JQuery function code. However, the problem was actually in server itself. I would think these errors need to reported as soon as they occur.

I have modified this code in my copy as follows. Anyone see the problem with this?? I would really appreciate someone's review since I am very new to this module.
Note : This code does not implement the fix for my point 1 above.

Thanks in advance.


PROCEDURE handleRequest(poRequest, poProps)
*
  • Analyze the requested URL to obtain:
  • a) VERB
  • b) Resource controller
  • c) Resource's parameters
    *
    LOCAL cUri,cRESTUri,cVerb,cResource,cParams,i,j,cController,cBaseURL
    LOCAL oDCH,oController,cHTMLOut
    Local lcPhysicalpath,llError
lcPhysicalpath = ""
cHTMLOut = ""
llError = .F.

IF VARTYPE(poRequest)<>"C"
cUri = poRequest.serverVariables("SCRIPT_NAME")
cVerb = poRequest.serverVariables("REQUEST_METHOD")

lcPhysicalpath = poRequest.serverVariables("APPL_PHYSICAL_PATH") && Phyical path to activate external debugger
ELSE
cUri = poRequest
cVerb = "GET"
ENDIF

j = 0
cResource = ""
cController = ""
FOR i = 1 TO THIS.Controllers.Count
cResource = THIS.Controllers.Keys[i]
cController = THIS.Controllers.Values[i]
IF ATC(cResource, cUri) > 0
j = ATC("/" + cResource, cUri)    &&ATC("/" + cResource + "/", cUri)
IF j > 0
 EXIT
ENDIF
ENDIF
ENDFOR

Do While .T.
  IF EMPTY(cResource)   && There is no controller for the specified resource
     cHTMLOut = "restHelper: unhandled resource on REST call (" + cUri + ")"
     llError = .T.  
     Exit   
    *RETURN "restHelper: unhandled resource on REST call (" + cUri + ")"
  ENDIF 
  IF j = 0   && Invalid REST call
     cHTMLOut = "restHelper: unrecognized REST call (" + cUri + ")"
     llError = .T.  
     Exit   
   *RETURN "restHelper: unrecognized REST call (" + cUri + ")"
  ENDIF

  cBaseURL = LEFT(cUri, j - 1)
  cRESTUri = SUBSTR(cUri,j)  && /server/resource/params --> /resource/params
  cParams = SUBSTR(cRESTUri,LEN(cResource) + 3)

  * Convert the parameter list to a collection
  LOCAL ARRAY aParams[1]
  LOCAL nCount, cPAram
  LOCAL oParams AS Collection
  nCount = ALINES(aParams, STRT(cParams,"/",CHR(13)+CHR(10)))
  oParams = CREATEOBJECT("Collection")
  FOR EACH cParam IN aParams
   IF !EMPTY(cParam)
    oParams.Add(cParam)
   ENDIF
  ENDFOR


  * Use the verb + parameter count to deduce the actual action
  * to be invoked on the controller
  *
  LOCAL cAction
  cAction = ""
  DO CASE
     CASE cVerb == "GET" AND oParams.Count = 1 AND LOWER(oParams.Item[1]) == "info"
          cAction = "infoAction"

     CASE cVerb == "GET" AND oParams.Count = 1 AND LOWER(oParams.Item[1]) == "help"
          cAction = "helpAction"

     CASE cVerb == "GET" AND oParams.Count = 0
          cAction = "listAction"

     CASE cVerb == "GET" AND oParams.Count > 0 AND !(ISALPHA(oParams.Item[1]))
          cAction = "getAction"

     CASE cVerb == "POST" AND oParams.Count = 0
          cAction = "addAction"

     CASE cVerb == "POST" AND oParams.Count > 0
          cAction = "updAction"

     CASE cVerb == "DELETE" AND oParams.Count = 0
          cAction = "zapAction"

     CASE cVerb == "DELETE" AND oPArams.Count > 0
          cAction = "dropAction"

     OTHERWISE
          cAction = oParams.Item[1]    &&"customAction"
  ENDCASE
  IF EMPTY(cAction)
     cHTMLOut = "restHelper: unrecognized REST call (" + cRESTUri + ")"
     llError = .T.  
     Exit   
   *RETURN "restHelper: unrecognized REST call (" + cRESTUri + ")"
  ENDIF

  * Create a instance of the resource's controller
  IF NOT FILE(cController)
     cHTMLOut = "restHelper: controller file <b>" + cController + "</b> could not be found"
     llError = .T.  
     Exit   
     *RETURN "restHelper: controller file <b>" + cController + "</b> could not be found"
  ENDIF

  oDCH = Newobject("dch",lcPhysicalpath+"prg\dch.prg")
  oDCH.initWithClass(cResource + "Controller", cController)
  IF NOT oDCH.createInstance()
     cHTMLOut = "restHelper: controller <b>" + cController + "</b> has some errors:<hr>" + FILETOSTR(FORCEEXT(cController,"TXT"))
     llError = .T.  
     Exit   
   *RETURN "restHelper: controller <b>" + cController + "</b> has some errors:<hr>" + FILETOSTR(FORCEEXT(cController,"TXT"))
  ENDIF

  oController = oDCH.Instance


  TRY
   oController.Request = poRequest
   oController.Props = poProps
   oController.Params = oParams
   oController.homeFolder = ADDBS(JUSTPATH(cController))
   oController.baseURL = cBaseURL


   IF PEMSTATUS(oController, cAction, 5)
    cHTMLOut = EVALUATE("oController." + cAction + "()")
   ELSE
    cHTMLOut = "restHelper: the <b>" + PROPER(cResource) + "</b>'s controller does not implement the requested action (<b>" + cAction + "</b>)"
    llError = .T.
   ENDIF

  *------+ Sept 2013    ... NEVER send any data type other than a string.   
  If VARTYPE(cHTMLOut) != "C"       
    cHTMLOut = "restHelper: the <b>" + PROPER(cResource) + "</b>'s controllers action <b>" + cAction + "</b> did not return character data type."
    llError = .T.
  EndIf

  CATCH TO ex
   cHTMLOut = "restHelper: error while processing the request <b>" + cRESTUri + "</b>:<hr>" + ;
              ex.Message + "<hr>" + cController
    llError = .T.           

  FINALLY
   oController = NULL
   oDCH.releaseInstance()

  EndTry

  *-----+ Get out
  Exit
EndDo
If llError
If INLIST(_VFP.StartMode,3,5)
   *------+ Abort to COM client (i.e. ASP)
    COMreturnerror(cHTMLOut,_VFP.SERVERNAME)
Else
   *-----+ Give error message but keep going.
   cHTMLOut = STRTRAN(cHTMLOut,'<b>','')
   cHTMLOut = STRTRAN(cHTMLOut,'</b>','')
   MESSAGEBOX(STRTRAN(cHTMLOut,'</br>',CHR(13)+CHR(10)))
endif   
Endif
RETURN cHTMLOut
*
ENDPROC
Oct 9, 2013 at 4:44 PM
Hi Titu1

Thanks for sharing your testings and enhancements.
1)
<< e.g. if we have two controllers say 'employee' and 'employeeCodes' then it may select 'employee' controller all the time >>
If you named the restful file 'employeeCodes.prg' then the first line should say:
DEFINE CLASS employeeCodesController AS restController
doing so should solve your point nº 2), but I'm also still testing
Developer
Oct 9, 2013 at 5:43 PM
Hi Karl,

Thanks from sharing.

Regarding the issue you commented on.

It's little more than the class name. I've registered this as an issue here. It replicates my problem in codeplex site (showcasing AVFP RESTful module).

The problem comes from this line. And it appears that the solution is also in that line as comments.

j = ATC("/" + cResource, cUri) &&ATC("/" + cResource + "/", cUri)

So now I hesitate to change this code since I do not know what else I might break.

Regarding point 2 of my email. This is a known issue but shit happens. My intent was to raise this error in server itself rather than pass it to the client side. See the code in point 3, specifically,
If VARTYPE(cHTMLOut) != "C"       
    cHTMLOut = "restHelper: the <b>" + PROPER(cResource) + "</b>'s controllers action <b>" + cAction + "</b> did not return character data type."
    llError = .T.
  EndIf
and then trap this error in COMReturnError().
Oct 9, 2013 at 9:20 PM
Hi Titu1

Thaks for your response

I can confirm both bugs , and I could add a third one:

http://www.rootsofmine.com/avfp603/customers/getemployees

sends incorrect response if a getemployees controller exist in the same folder ( getemployees.prg )

seems that controllers are getting high precedence over actions , and also does not trim names , so ..

http://www.rootsofmine.com/avfp603/customers/getemployeesXX

launches getemployee controller in the same 'wrong' asignment
Oct 9, 2013 at 10:25 PM
Hi again,

Titu1 could not test the great fix proposed ( lazy on recompile now) , but I vote for remove the default ( emty) action for the controller
  IF EMPTY(cAction)
     cHTMLOut = "restHelper: unrecognized REST call (" + cRESTUri + ")"
     llError = .T.  
     Exit   
   ENDIF
Could be usefull to have a shorter link to call a default sub-site or any other utility.

Thanks in advance
Coordinator
Oct 10, 2013 at 12:11 AM
I encourage you all to make all of the changes to the source and recompile everything and test. The instructions are in the video. I can help in this forum as long as you all do most of the work.
Oct 10, 2013 at 12:39 PM
oh oh, My first day here and found myself testing bugs , reviewing code, recompliling and thinking how to fix, I dont blame at all but.... my bad luck is dramatically growing minute by minute :)

Anyway, lets take it with humor

Titu1, You have a fantastic idea placing some debug capture there,

I think I found at least part of the bugs, but my fox coding is a bit oxidized, so...

Codeplex dont have PM's ? Upss

the other alternative is....

Claude can we contact Victor to ask for a specific quick fix ?
Editor
Oct 10, 2013 at 5:25 PM
Hi KARL_U,

re:
Claude can we contact Victor to ask for a specific quick fix ?
IMHO, one of the beauties/advantages of AVFP is that all of the source code is available to us so we can modify it if we so wish.

I'm personally using AVFP to develop RESTful like services. I'll be honest, I've only used "resthelper" as a code reference and have modified my own copy of the ActiveVFP.dll to allow me to create and call my own "RESTful" handler(s) i.e. alternatives to using "resthelper".

I only did this because I favored a slightly different way of doing things and unfortunately found the "resthelper" code a bit too restrictive for what I needed to do.
This is not a criticism of the work that's already been done. In fact, without it, I would most probably never have got started.

I've been meaning to publish details on how I'm doing things for some time now. I'm also intending to create an new issue - which is really a proposal - detailing my suggested changes to proxystub.prg which would enable everybody to create their own preferred versions of "RESTful Handlers". Unfortunately, I've been struggling to find the time lately.

I'll try to do this very soon though. As a starting point, please use the following link to view a schematic which hopefully demonstrates and compares how things work now and how they would do in my proposed modified version...
ActiveVFP RESTfulAPI Extensible

I hope that this all makes sense and is of interest.

P.s. Here's something that pretty well reflects my way of thinking...

URI-design rightly remains an art not a science

Coordinator
Oct 10, 2013 at 5:33 PM
Yes, please contact Victor. That's what we're all here for. He will more than likely help as much as he can, but, no one who wants to improve AVFP should shy away from making changes to the source and testing - it's all just VFP after all.

RESTful is fairly new and undoubtedly can be improved. I even have 2 requests for it (which I did in the recent video):

1.) use the Session Class to make sure settings are correct for custom code
2.) put an Error handler in with COMReturnerror to make sure little mistakes don't take down the server.
  • AVFP scripting (placing code in .avfp files) already implements these which is why people probably think there are less problems with it.
BTW, I'm wondering if some of what you all are experiencing is related to the above 2 items
Oct 10, 2013 at 8:24 PM
SimonCi and Claudefox , many thanks for your patienced and kindly explanations.

I understand your points of view and the philosofy behind the importance of users contribution to the project, in fact is a very positive incentive to create/improve a lot of new features ( a factory of ideas called some years ago). and I believe AVFP have a great potential because it's power and easy to install , also fast modular product build tool.

What is not so clear to me is something I'll try to explain in my 'best' english, but please do not consider it criticism, but instead a constructive oppinion ( remember I am the newbie ). :)

The Restful controler will be the 'second core' of AVFP shortly and should be the primary soon because of the flexibility it aports ( call a rest from anywhere including res to rest calls etc etc)
Knowing so, and the subyacent importance, my unclear part is..... why the team of coordinators and developers 'encourage' to give this task to us ( inexperted in controllers development ) . In terms of quality and client satisfaction ,It is supossed to be ready to operate and fixed by expert profesionals prontly. even with 2 pending improvement requests and having the prg owner in the developper team... humm sorry but that make no much sense for me

ok ok, it could be remaked in the future, but the confiance in thinking that the important pieces are robust ( althoug need to be enhanced) is important.

I think those are the main reasons , because we are reticent to try to fix something important that would affect someone's critical app bussiness, cause our fixes can produce other bugs even worse. ( not the same case as it were a new not critical module in progress , note:: a pair of those would animate some of us to re-learn vfp oop ).

Well, that's just my own feeling , and hope I've stated to give contributions when my spare time give me signed autorization :)
Developer
Oct 10, 2013 at 9:12 PM
Hi Karl,

The two issues you mention seen to be related to the same problem. i.e.

j = ATC("/" + cResource, cUri) &&ATC("/" + cResource + "/", cUri)


In my view, this URL parsing portion needs to be re-visited. Assuming we only allow have URLs in simple '/application/Controller/action/parm/parm' ( or.. '/Application/controller' etc, etc). Then it'll be simpler to get controller/action by switching from the using the currnent. ...

cUri = poRequest.serverVariables("SCRIPT_NAME")

.. to Something like..

lcRestCall = StrTran(ADDBS(poRequest.serverVariables("PATH_TRANSLATED")), poRequest.serverVariables("APPL_PHYSICAL_PATH"),"")

Then lcRestCall will be in the format of....

controller\action\param\parm

The above can then be easily parsed into REST components based on '\' as delimiter. I'll try something on those lines in my local copy, when I get back to testing.

BTW... I too feel I may break something in codeplex hence my reluctance.
Oct 10, 2013 at 9:39 PM
Opss, and Titu1, my apologizes for spaming your post.

I will try to describe what I found .

You are more than correct, it's in the lines containing ATC command ( all of them or almost)

the first one ( line 98 in original prg )
IF ATC("/" + THIS.Controllers.Keys[i], cUri) > 0 &&IF ATC("/" + THIS.Controllers.Keys[i] + "/", cUri) > 0

i think is incorrect because search a folder name (controller) in the whole Uri string beguining the search from the first slash and should be between slashes to avoid encounter an Action name with identical name as a controller ... so.shoud read. something like :
 IF SUBSTR( cUri , ( RTOC( "/", cUri , 2)  , ( RTOC( "/", cUri , 1) +1 -  RTOC( "/", cUri , 2)) == THIS.Controllers.Keys[i]
remember we need to compare exact strings also , to avoid words with same root ie, sample , sample001
please help with this, I cant remember the las time I used string concatenation commands :)
could be simpler pacing positions in vars to make the STR

and so on.. on other ATC's cResource matching between slashes and cAction at right of last slash

( what I am not sure is if it reads more levels of the folders tree root beguining in rest/controllers folder )
Coordinator
Oct 10, 2013 at 9:42 PM
Come on guys, we have copies of everything, so there is no way you're going to break anything.

I agree the RESTful Controller/MVC model is very important but the only way this project goes forward is for USERS to take it forward. I am only an advisor at this point. Give me the improvements (after testing, of course) and we'll produce a new release.

There's no "smoke and mirrors" there - it's all Foxpro!!
Developer
Oct 10, 2013 at 10:33 PM
Hi Karl,

I doubt we can reach a stable solution if we only look at slashes e.g. all these are valid calls

/application/Controller/action/parm/parm
/application/Controller
%2Fapplication%2FController
/application/applicationsubfolder/controller ( parent WS)
Developer
Oct 10, 2013 at 10:55 PM
Hi Simon,

That looks very impressive. I was also thinking of desiging something like the following example,, with authentication and authosization built into each controllers

e.g.
/application/ParentController/UID/ChildController/action/param.
  1. in the above URL , we have a parentController that has to give approval to access the child controller. It also has authorization filters to process either/or/and/ the GETs & POSTs ( and put??) actions.
  2. UID is the master key of the Parent table attached to the controller. This is hashed.
  3. ParentController then calls the ChildController which replicates the functionality of the parent and will (in the above case) create a response()
This approach is scalable since this URL can be extended to call multiple child controllers to replicate database relational schema.

Maybe, something you may like to think over?
Oct 10, 2013 at 11:08 PM
Hi Titu

I thik it should ( if perform the search right to left) , last pair is stored in a dictionary of pairs also. and compared only withing it.
The left most part of the Uri ( at the left of 2º slash counting from right) is stored in another dictionary.. and no matter if is longer or shorter.

If you see the list of posible defined Actions are singles... so your first url will never happen
the second is send as error because controller not found ( aplication) although it stores 'Controller' as an Action
for your 3rd there is a slash replacement somewhere
and 4º...is same as 2º applicationfolder as controller and controller word as action... very few probabilities to occur a coincidence of two names

So... let's try .. nothig to lose,

in my last post the ) + 1 - RTOC should be )- 1 - RTOC

allways confused me subtract 1 from lower RTOC OR add 1 to higher ( but never both)

in case of cAction :
LEN(cUri ) - ( RTOC( "/", cUri , 1)
Developer
Oct 11, 2013 at 12:00 AM
Hi Karl,

Ok, Like you said. Nothing to lose. Let us know how it goes.
Oct 11, 2013 at 12:02 AM
Hi Titu1

I thought trhere are more ATC lines , just 3 , and I lived the last untouched because if greater tan zero exit,... so if is suposed to be just one slash, then the command ATC is correct

the right command is RATC :) :)
I' send you in the next post the lines from 98 to 139... for testing on a copy of the original and your fix version.
I know how to recompile into a fll, but I don't know the rest of the process, where an how to substitute
Thanks
Oct 11, 2013 at 12:03 AM
   IF SUBSTR( cUri , ( RATC( "/", cUri , 2)  , ( RATC( "/", cUri , 1) -1 -  RATC( "/", cUri , 2)) == THIS.Controllers.Keys[i]
    lHasController = .T.
    EXIT
   ENDIF
  ENDFOR
  IF NOT lHasController
   RETURN .F.
  ENDIF
  
  * If we get through here, then the uri MAY be a REST api call
  RETURN .T.
  *
 ENDPROC
 
  
 * handleRequest
 * Handle a given HTTP request
 *
 PROCEDURE handleRequest(poRequest, poProps)
  *
  * Analyze the requested URL to obtain:
  * a) VERB
  * b) Resource controller
  * c) Resource's parameters
  *
  LOCAL cUri,cRESTUri,cVerb,cResource,cParams,i,j,cController,cBaseURL
  IF VARTYPE(poRequest)<>"C"
   cUri = poRequest.serverVariables("SCRIPT_NAME")
   cVerb = poRequest.serverVariables("REQUEST_METHOD")
  ELSE
   cUri = poRequest
   cVerb = "GET"
  ENDIF
  
  j = 0
  cResource = ""
  cController = ""
  FOR i = 1 TO THIS.Controllers.Count
   cResource = THIS.Controllers.Keys[i]
   cController = THIS.Controllers.Values[i]
**   IF ATC(cResource, cUri) > 0
      IF SUBSTR( cUri , ( RATC( "/", cUri , 2)  , ( RATC( "/", cUri , 1) -1 -  RATC( "/", cUri , 2)) == cResource
**    j = ATC("/" + cResource, cUri)    &&ATC("/" + cResource + "/", cUri)
      j = ATC("/" + cResource, cUri) 
Oct 11, 2013 at 12:46 AM
Victor , te quedas sin la media cerveza caliente de invitacion por hacerme revisar el código.... lo sepas ;) :P

Saludos semi-cordiales mudito
Developer
Oct 11, 2013 at 4:21 AM
Hi Karl..
Tried.. but syntax error
SUBSTR( cUri , ( RATC( "/", cUri , 2) , ( RATC( "/", cUri , 1) -1 - RATC( "/", cUri , 2))
Developer
Oct 11, 2013 at 5:04 AM
Edited Oct 11, 2013 at 5:14 AM
Hi Claudefox,

Regarding ... __Give me the improvements (after testing, of course) and we'll produce a new release....
__
I am giving my version of restHelper.prg.

1) I've removed the looping in the controllers directory, to locate the controller prgs,, in both the problem isREST() and handleRequest() functions. This would improve the speed if any user creates a large number of controllers.

2) Most errors would now be trapped in comreturnerror().
oDCH.createInstance() had to be excluded from the error trapper.

3) Regarding....

__..... use the Session Class to make sure settings are correct for custom code
........
• AVFP scripting (placing code in .avfp files) already implements these which is why people probably think there are less problems with it.__

I saw this in your video, and the problem it immediately created.

... AVFPHandler runs under session class. If you make resthelper also as a session class, then any tables opened in this RESTFul class, would not be readable in the AVFPHandler session ( so the AVFP view, in your video, returned zero customer records).

Since it impacts existing avfp installations, I've left it out. However, it will be a good thing to have resthelper as session.

4) My code is a follows.

OK.. I cannot paste it here. I'll try to attach it to the issue.

The file is attached under this link (https://activevfp.codeplex.com/workitem/34535)
Oct 11, 2013 at 5:15 AM
ouch, sorry Titu1
I should have tested it in vfp command window before posting
try this ones already tested, line nº 98 and 137 respectively .... crossing fingers :)

IF SUBSTR(cUri,RATC("/",cUri,2),(RATC("/",cUri,1)-1)-RATC("/",cUri,2)) == THIS.Controllers.Keys[i]

IF SUBSTR(cUri,RATC("/",cUri,2),(RATC("/",cUri,1)-1)-RATC("/",cUri,2)) == cResource
Developer
Oct 11, 2013 at 2:51 PM
Thanks Karl,

We'll still need to loop thru all the controllers every time, even when we are not making RESTful calls. For restfull we loop twice.

This can hit performance, when the list of controller grows. I am looking at a highly normalized database, that has couple of thousand tables. Theforefore, if the client side developers are using entity framework, they may insist on highly normalized controllers. VFP file search can be optimized by windows file indexing but doing a "for.. endfor" loops can not be optimized.

In my view, we should eliminate the need to run loadControllers function in RestHelper altogether. This can now be done, if my proposed fix is accepted.

Have a look at my fix. Try breaking that code.
Oct 11, 2013 at 4:48 PM
Hi Titu1

That sounds like a great improvement, i am doing thigs step by step, and the logic is .. once a module is full-proof, then we can improve it ( otherwise the bugs coud be growing exponiental).

well, you are right a seek of file names in a big controllers tree with hundreds of folders should take some time , and a double pass is a bad idea, so I'll try your fix for sure thanks for doing the hard work.

I'm wondering about some other improvements regarding debugging that would fit my needs ( and probably many others), something like ... storing requests and responses in a semi-separate 'log console' module , a Memo field in a dbf. that could be consulted remotelly and avoid the need of in-site-debug of that part. would be activated/deactivated on demand. and configured in a way to be fired and log ( or not) requests for itself .

That kind of utillity would permit debug and/or 'Admin' AVFP's apps installed on comercial Hostings. ( other functionality than a simple log could be added, but then need more security considerations )

Do you think it is possible with the actual config ?

I have an abandoned/unfinished jquery UI web interface that simulates a vfp desktop, and would fits perfect in the future for a console like that.
Developer
Oct 11, 2013 at 9:52 PM
Hi Karl,

If you have just started playing with this system, than, I would suggest you get little more familiar with the documentation. I found it very good for learning AVFP internals and evaluating it as a viable solution to my needs.

AVFP already has a debugger as a memory dump.

And yes, I think for 6.03, Claudefox is working on making debugger work with AVFPHandler. If not, then you may have to create your own debugger, ..something outside of IIS COM. I have a rickety one that requires killing this process, in task manager, after I hit any unhandled exception. But it does trace me to the problem line.

I am sure others have made their own too. Do not go on this path of creating a debugger, unless you are sure, AVFP meets your needs or without getting a better handle on this aplication.

Also, try not to fork out too soon, it is a pain if you have to subsequently upgrade(i.e. if something new comes up in AVFP and you want it in your apps) or you want to give back some prg to the community.
Oct 12, 2013 at 2:32 PM
Thanks Titu1

I totally agree on avoiding modify parts that are going to change on new releases.

Having a hard time upgrading to .NET Framework 4.0 , and an easy one recompiling. .... regarding the fixes...are you sure that your resthelper version haven't dependencies on other new enhancements ? debugging lines maybe ? I say it because... I might be doing somencing wrong or... tha's the case, getting the same error screen as if the dll werent registered.

( another good reason to not let users improve the main parts... dependencies) and when doing in separate modules keepeng trak of possible incompabilities.

Anyway... still testing the 2 lines fix.
Oct 12, 2013 at 5:00 PM
Hi Titu1

Just one last thing before return to real life projects

In the actual situation controllers searching is performed 'allways' before any normal Uri ( except the ones containing parameters prededieng with a '?')
So it ralentizes a few ( a lot if many controllers and folders are inside the tree), I propose checking more standardt Uri's to
' jump' the controllers searching procedure.

.AVFP extension is the default one, but .more ones can be added .ASP .HTM .HTML .DOC .PDF etc
  * Basic syntax check
  IF !EMPTY(JUSTEXT(cUri)) OR AT("?",cUri)<>0  OR  RATC(".AVFP",cUri,1) == LEN(cUri)-4
   RETURN .F.
  ENDIF
Hey! that can speed up even the load of default.avfp sample page ( i hope) :)
Developer
Oct 12, 2013 at 7:44 PM
Edited Oct 12, 2013 at 8:32 PM
I think the problem is that we don't have good documentation on RestFul module. RESTful module in AVFP gives a very basic framework. It is expected that anyone wishing to use it will make their own routers. I think Simon is thinking of enhancing this portion.

This is my interpretation on what's already in AVFP REST module. I may be wrong and open to correction. However, I hope it helps users new to this module.

1) All the controllers are placed in ~\rest\controller folder... as Prgs files only. ( I.e. subdirectories should Never exist.).

2) REST calls must always be extensionless. i.e. no AVFP, HTML, PDF etc). Otherwise, it cannot be called as RESTful call. Similarly, you cannot sent any parameters in RESTful calls (i.e. ?parm&value.)

e.g. A normal non-RESTful as follows

/Server/sample.avfp?uid&100;Name&Testing

would be sent in RESTful as ( sample is the controller)

/Server/sample/100/testing


However, in my view, named parameters can still be sent in Restful calls without violating RESTful norms e.g.
Server/sample/uid=100/Name=Testing

Your routing, just needs to understand and parse these parameters properly.

3) No Authorisation module exists in AVFP RESTful module. Normally, if any REST call is unauthorised ( this is different from unauthenticated), the response should be 'blank'. However, it would be easy to incorporate the authentication from non-REST portion of AVFP. Or.. make your own.


4) AVFP also does not have a standardized error handler. Again it is expected that users would make their own.

Currently, this seems to create problems when new AVFP users are just 'developing' the RESTful application, out of the box, and, happen to get RunTime errors. This is due to lack of debugger.

Normally, an error handler would send the users to a friendly page in none-REST( or standard Http error in REST), in production versions.

In development version, it should raise a comreturnError() to help with debugging/development.

.............
So, unless I misunderstood your suggestion, I would not recommend you sending any folder names in RESTful calls. That may be outside what REST is about.

Also, due to extensionless nature of calls, the following is sufficient for the existing AVFP router

IF !EMPTY(JUSTEXT(cUri)) OR ......
Oct 12, 2013 at 10:07 PM
Hi Tytu1
Many thanks for taken your time on such detailed explanations.really appreciated

Just asked because I get same errors calling normal Uri's in root app, i.e hello.avfp but I will re-test all and keep you informed

I am aware of most of what you are describing
IF !EMPTY(JUSTEXT(cUri)) OR .....( just let me test if demo.avfp is part of the search controllers procedure I was only saying to do the jump for docs in the app root , not to open them in the controllers folders),

Coincide in your view about rest and debugging standarization, and you have mentioned the second reason of my view regarding what users improve/create to meet their own needs, That is exactly what is all for!

I could be wrong in my theory but...
If users have a small resume/guide of the new release in progress..(alpha?, beta? who knows? ).... can create modules commented and/or prepared to be adapted in the future , but creating for throw to the trash ,or review all after upgrading is not the best incentive. so many users (unless a few :) ) make fixes , patches , mini-tools and custom adaptations where needed and that's not a good material for share.

To be honest that is exactly what I am going to do ( If I have success, and I'm in the urgent need to reduce debugging time ) with a mini-log for restful, but I promise that if I encounter the time to standarize a module .. I'll share it.
That does not put apart my priority list over the importance of things, and try to fix semi-important bugs ( under my reduced knoledge) that could affect many other users as better and soon as possible. ( even if I can live with it using a careful naming convention)

ok I'm boring you all, :) time to meet the server. Thanks again for all the help
Developer
Oct 12, 2013 at 11:15 PM
See here for your email dated oct 8th,
You mentioned moving some folders to speed up AVFP. Could be the cause of bugs.
Oct 12, 2013 at 11:50 PM
Didn't mention bugs at all, just that calling from restful pages on the root app ( in my particular experience) is slower than moving it to a different folder.... excuse my if I can't explain things better , english is not mi mother language.
Editor
Oct 13, 2013 at 1:49 AM
Hi KARL_U & Titu1

Goodness me guys, I can't keep up with you two - Good on ya! :-)

I'll try to add my bits when I can if that's OK with you both.

To start with re:
KARL_U wrote:
Hi Titu1

Just one last thing before return to real life projects

In the actual situation controllers searching is performed 'allways' before any normal Uri ( except the ones containing parameters prededieng with a '?')
So it ralentizes a few ( a lot if many controllers and folders are inside the tree), I propose checking more standardt Uri's to
' jump' the controllers searching procedure.

.AVFP extension is the default one, but .more ones can be added .ASP .HTM .HTML .DOC .PDF etc
  * Basic syntax check
  IF !EMPTY(JUSTEXT(cUri)) OR AT("?",cUri)<>0  OR  RATC(".AVFP",cUri,1) == LEN(cUri)-4
   RETURN .F.
  ENDIF
Hey! that can speed up even the load of default.avfp sample page ( i hope) :)
Yep, I agree with you there KARL_U - as per my schematic - non-extensionless requests ALWAYS resulting in call to resthelper 1st as part of the existing execution sequence.
I've modified my own copy of "server :: process" to wrap an "IF" statement around the "resthelper" code block as follows...
        *.....................................................................................................
        IF EMPTY(IIF(AT('?',oProp.ScriptPath)=0, ;
                    JUSTEXT(oProp.ScriptPath), ;
                        JUSTEXT(SUBSTR(oProp.ScriptPath,1,AT('?',oProp.ScriptPath)-1))))
            *.....................................................................................................
            * We are going to use Victor Espania's original/default RestHelper class as the RESTful Handler

            **************************************************************
            * V:1.0 REST-like APIs implementation code block/logic
            * Author: Victor Espina
            * Date: April 2012 (original v:1.0)
            **************************************************************
            LOCAL oRESTHelper

            * Instantiate RESTful Handler
            oRESTHelper = NEWOBJECT("restHelper","PRG\RESTHelper.PRG")

            * Set/store folder location of application root in RESTful Handler object.
            oRESTHelper.setFolder("ROOT", cRootPath)
            * Set/store folder location of controllers.
            oRESTHelper.setFolder("CONTROLLERS", ADDBS(cRootPath) + "prg/rest/controllers")

            * Set/store list of controllers found in above defined "controllers" folder 
            * as an array in the RESTful Handler object.
            oRESTHelper.loadControllers()

            * Use RESTHelper to check if the request - according to RESTHelper rules - is a REST-like API call.
            * Then, if REST-like API call, use RESTHelper to call corresponding controller which, when finished,
            * returns the response.
            IF oRESTHelper.isREST(oRequest)
                oProp.appStartPath = oRequest.serverVariables("APPL_PHYSICAL_PATH")
                RETURN oRESTHelper.handleRequest(oRequest, oProp)
            ENDIF
            **************************************************************
        ENDIF
        *.....................................................................................................
Sounds like a pending proposal to me!

And yes, as you've most likely spotted, I'm using "query strings" to handle parameters. Twitter's way of doing things has greatly influenced the way I'm handling REST requests.

I will hopefully get some more time later to post some more responses. As I said, I'm finding it difficult to keep up with you two :)
Oct 13, 2013 at 12:37 PM
Hi SimonCi

Many thanks for sharing that code, that helps a lot to put all puzzle pieces together and finaly fix it



Titu1 You misunderstood my comments on the established system improvements, which of course, does not refer to you, and never applies when trying to fix a major problem.

your work and attitude are commendable and I never intended to offend in any way, I sincerely appreciate all the effort and support you have given to solve this exceptional situation.

Thank You
Developer
Oct 13, 2013 at 5:01 PM
Edited Oct 13, 2013 at 6:06 PM
Thanks Simon. That clarified matters since I was on a entirely different page with Karl.
And Karl, please do not think you or anyone can offend me here. We are all learning and just trying to share that learning. I am really sorry you felt that way.

A small observation on Simons approach,

In my opinion,
1) a extension less call would be a RESTful call. This is undisputed by all
2) As per Victor, in addition to above it should not use the '?' for parameters. This is the version in AVFP.
3) As per Simon, we can use it for query strings.

I assume, this would be in Simons version of RESTFul 'extensions'. I guess each approach has it's benefits.

Now, everyone we talk to, seems to have their own manner of forming RESTFull calls. The important point is that these need to be documented during programming. Victor did an excellent job with this module, He also mentions it in compact manner as part of this comments.

So, for AVFP upgrade, if we are to cover widest range of opinions, we can only use the following function (i.e. point 1 above, since this is accepted by everyone as base of RESTful call),

IF !EMPTY(JUSTEXT(Uri))

However, my suggestion is that, we ignore the URL altogether and attempt to locate the controller directly using file system.

The function to check RESTful call or to find the name of the controller may run something on these lines
 *-----+
  Local lcRestCall, lnRows, laParseRest(1), lcControllerPath,lcPath
  lcRestCall = StrTran(ADDBS(poRequest.serverVariables("PATH_TRANSLATED")), poRequest.serverVariables("APPL_PHYSICAL_PATH"),"")       
  If EMPTY(lcRestCall)
    *---- Error.. pigns only
    Return .F.
  EndIf
  *-----+ lcRestCall is now as controller\action\param or Form.AVFP
  lnRows = ALINES(laParseRest, lcRestCall, .F., "\")
  If !EMPTY(JUSTEXT(laParseRest[1]))    
    *---+ This is not REST call
    return .F.
  endif
  lcPath = THIS.Folders.Values["CONTROLLERS"]  && ..Pending.. change this using server variables if called outside the resthelper class. 
  lcControllerPath = lcPath+laParseRest[1]+".prg"
  *------+ Check if this controller exists
  If !FILE(lcControllerPath)
    Return .F.
  endif
  rETURN .t.    
Editor
Oct 13, 2013 at 7:05 PM
Hi Titu1 (don't your name btw)

Just in case we are talking at cross-purposes...

Titu1 wrote:
In my opinion,
1) a extension less call would be a RESTful call. This is undisputed by all
Yes totally agree! Hopefully anybody that doesn't agree will shout and let us know.

Regards,
Simon
Editor
Oct 13, 2013 at 8:05 PM
Following on...

Titu1 wrote:
In my opinion,
...
2) As per Victor, in addition to above it should not use the '?' for parameters. This is the version in AVFP.
Not sure that I understand you on this point? Are you saying that REST requests should not use '?' and query strings. If so, not sure that I agree with you here but as you say, resthelper in AVFP version 6.03 doesn't recognise/accept a request as being RESTful using this type of syntax i.e.
PROCEDURE isREST(poRequest)
  
   ....
 
  * Basic syntax check
  IF !EMPTY(JUSTEXT(cUri)) OR AT("?",cUri)<>0
    RETURN .F.
  ENDIF
and this is one of the reasons why I'm not using resthelper as my chosen "RESTful Handler". It's just my choice.
Editor
Oct 13, 2013 at 8:36 PM
Just to clarify about purpose of;
IF EMPTY(IIF(AT('?',oProp.ScriptPath)=0, ;
                    JUSTEXT(oProp.ScriptPath), ;
                        JUSTEXT(SUBSTR(oProp.ScriptPath,1,AT('?',oProp.ScriptPath)-1))))
which I discussed above.

This is suggested only to shave off a bit of processing time when requests are not RESTful according to resthelper logic.
Hopefully you agree - unless I've lost the plot - that requests like the following in AVFP 6.03 invokes resthelper;
Can't see the point of resthelper running at all with these types of URI's hence the "IF" statement.
Wasn't this the point you were making KARL_U?

BTW, all of the "comments" in the code snippet that I posted earlier were not included for the purpose of educating others what the code did. They are all comments that I made to myself back in August when I was first trying to work out how AVFP worked - they just got included when I copied/pasted the code block.

I promise, I wasn't trying to patronize or teach anyone how to suck eggs :)
Oct 13, 2013 at 9:05 PM
Hi SimonCi and Titu1

I totally agree on that point.

Just to clarify avoiding mixing concepts.

In my view, there are 2 steps, but joining those into one is open to discussion

1) An official "patch" for current stable release should be adopted ,and separate documented (in a way that does not modify much the actual behaivour but fix everything) so that old/new users can feel comfortable without worring about internals.

2) Upcoming proposals/enhancements on restful module. ( I leave that part to you all)


Seems to be that the way I express may cause some misinterpretations, I will put more effort in order to prevent that from happening in the future

The restfull standart ussage is wide open, so please correct me if I'm sayng something really absurd.
A restfull controller response can be anything ( including another restfull call).

My questiong is... A call to a AVFP <script> page (as a controller response) could cause any type of conflict ?

SimonCi do you see any differences on that in curren ( bugs'free) version and yours ?


Thanks & Regards
Oct 13, 2013 at 10:59 PM
Hi all

Still fixig a minimum fix.
I thought I have found a way to avoid compare all extension names to skip , but found a very strange case that fails.

http://localhost/ActiveVFP603/folder.1/a
( rare to named a folder with a dot and a controller with just one character... but possible is)
  * Basic syntax check
  IF !EMPTY(JUSTEXT(cUri)) OR AT("?",cUri)<>0  OR  RATC(".",cUri,1) == LEN(cUri)-4
   RETURN .F.
  ENDIF
Ideas are wellcome


Simonci regarding:
Wasn't this the point you were making KARL_U?

Yes, exactly althoug I was also thinking more on my last question
think of it as a restful controller simple menu fashion calling normal ( no restful's) AVFP pages ( or anything else)
Oct 13, 2013 at 11:26 PM
well, I mean internal ( no user interaction so the response only fires once) , but in this case is a Uri with parameters http://localhost/ActiveVFP603/default.avfp?debug=on
Oct 14, 2013 at 12:52 AM
I think I found the way, RATC to the rescue again.

```
* Basic syntax check
IF !EMPTY(JUSTEXT(cUri)) OR AT("?",cUri)<>0 OR RATC(".",cUri,1) > LEN(cUri)-5 OR (RATC(".",cUri,1) > LEN(cUri)-5 AND LEN(cUri)-RATC("/",cUri,1) > RATC(".",cUri,1))
RETURN .F.
ENDIF
``` There is still one exception :
controllers on the root folder named with a dot in last 4 positions with default action ( no action in Uri) will be considered as no rest Uri's i.e.

http://localhost/ActiveVFP603/controll.er

Not sure if that exception compensates the list of actual and future? extensions names searched programatically.


caveats: extensions longer than 4 ? default.AVFPX
Oct 14, 2013 at 1:16 AM
Hi SionCi

Thinking a bit more about the official patch/fix , I hardly feel strong enough to suggest it but.... that almost implyes a new fixed release.( that new users do not need to recompile ) And that means also a full test process depending on how big the changes are.. etc

Titu1, what do you think about all this ?
Developer
Oct 14, 2013 at 1:36 AM
Hi Simon,

Regarding ..
Are you saying that REST requests should not use '?' and query strings
Not really, I was just stating what we currently have in AVFP. I doubt we'll ever have any global consensus in this issue. You are free to use "?",":" or blank.., all would be correct, If, your router can handle it and both client and server understand it.

If you agree with the above statement, then you have to also agree that we should not be using the value of ScriptPath, ....in any shape or form.... to.

1) Find if this is a RestFul call.
2) Find the name of the Controller.

My code snippet tries to do just that.

If you make it into a function and replace your following code ..
IF EMPTY(IIF(AT('?',oProp.ScriptPath)=0, ;
                    JUSTEXT(oProp.ScriptPath), ;
                        JUSTEXT(SUBSTR(oProp.ScriptPath,1,AT('?',oProp.ScriptPath)-1))))
with that function call, then it'll work with both AVFP code... and also with Twitter type calls..... with almost no reduction in speed.
Editor
Oct 14, 2013 at 4:50 AM
Titu1 wrote:
If you agree with the above statement, then you have to also agree that we should not be using the value of ScriptPath, ....in any shape or form.... to.

1) Find if this is a RestFul call.
2) Find the name of the Controller.
...
etc,etc...
Hi Titu1,

I'm afraid that I think that we are talking at cross-purposes here but I don't think this is a deal breaker anyway.

In a way, I'm thinking that this specific topic/issue doesn't really belong in this thread.

Please let me explain and please bear with me on this.

First of all, please forget the way RESTful requests are handled altogether for the time being.

Just one question for now please;

Do you agree that the following request is not RESTful as it currently stands in AVFP 6.03?
http://localhost/ActiveVFP603/default.avfp?debug=on

Ps. Please don't think me rude if I don't respond back to your answer on this until late tomorrow night because I'm out of my office most of the day tomorrow (UK time that is).
Developer
Oct 14, 2013 at 8:27 AM
Edited Oct 14, 2013 at 6:37 PM
Hi Simon,

I agree, this link is getting just too long. I guess that's what happens when REST is the topic of discussion. We need to end this now.

To answer your question. As in the current version of avfp...., it is not restful.
We've already established consensus on this.

So while, we are not discussing the manner of REST handling, may I point an issue I did not understand, and could be a potential bug in the following code, you have for twitter version.
IF EMPTY(IIF(AT('?',oProp.ScriptPath)=0, ;
                    JUSTEXT(oProp.ScriptPath), ;
                        JUSTEXT(SUBSTR(oProp.ScriptPath,1,AT('?',oProp.ScriptPath)-1))))
why are you searching for "?" in oProp.ScriptPath.
From what I know, oProp.ScriptPath is just a path to avfp file (or controller) and it does not have the 'parameter' portion of the uri.

Update:..
Hi Simon, I just re-read my reply. It sounds little rude which was not my intension. I apologise.
I have no knowledge on how the Twitter parameters are handled in server variable 'SCRIPT_NAME'.. and just getting familiar with AVFP . Therefore, I should not be making judgements like above.
Coordinator
Nov 6, 2013 at 11:57 PM
Hi to everyone.

I come a little late to this discussion; my apologies for that.

As claudefox said, RESThelper and MVC implementation in AVFP are new and therefore are mean to be improved in many ways. I my self had very little experience building REST webservices when I wrote RESTHelper. After creating a couple of REST services using WCF in .NET I would use a complete different approach now. In fact, I completely rewrote RESThelper after submiting it to claudefox's revision but he decided to keep the original version because it was already tested.

I liked a lot Simons proposoal for a way to personalize the handler of REST calls on the fly (I hope he used a delegate pattern for that). Anyway I'm very glad the RESTHelper is being used, discussed and used as inspiration to build better solutions for implemeting REST services in AVFP.

As claudefox said, that's the spirit here.


Regards


Victor Espina
Developer
Nov 7, 2013 at 3:55 PM
Hi Victor,
Nice to see to here. Also, I agree, REST would improve greatly with Simon extensions.