I just did a test about your program, and I found that an extreme vertex that obtained does not satisfy all inequalities. Note that I just test inequalities case (3D):
A=
-32.7680000000000 17.6400000000000 -3.20000000000000
-3.37500000000000 6.25000000000000 -1.50000000000000
17.5760000000000 2.56000000000000 2.60000000000000
79.5070000000000 10.8900000000000 4.30000000000000
32.7680000000000 -17.6400000000000 3.20000000000000
3.37500000000000 -6.25000000000000 1.50000000000000
-17.5760000000000 -2.56000000000000 -2.60000000000000
-79.5070000000000 -10.8900000000000 -4.30000000000000
b=
5
18
68
244
2
-11
-61
-219
One of the extreme vertices: [1.5530 4.9322 9.7233]
In order to test if this three values satisfy all inequalities, I multiply A by [1.5530 4.9322 9.7233]' to compare the new set of b values with the previous b values, and I found not all of new b values less than the original b values, which means they are not satisfied.

Thank you to provide this very useful toolbox. I have a question: if I obtain the rotation matrix from your method, then how can I get the coordinates of any point in the new coordinate system (target).

I am sorry about my mistake, and you are right. I should only expand upper values or decrease lower values to test this program, and it give me the results quickly. In order to have a better understanding of this program, could you provide some references to the algorithm used?
Best regards,
Cong

Dear Christophe,
No, changing tolerances won't help. Your problem is ill-posed. Because the ideal triangles only touch at a single point, small perturbations of the input data can make the triangles intersect at multiple points or not at all.
This is why, in the Description section above, I mentioned "It is to be emphasized that A,b must define a solid region". Your intersection point is not solid in R^2, so the inequalities [A1;A2],[b1;b2] are not a legal way of expressing it.

Dear Matt,
tried calculating the intersection between a two triangles touching each other in a point but got following error.
Assume can be solved by setting tolerances correct?
Thanks
Christophe
>> [A1,b1,Aeq1,beq1]=vert2lcon([2,0;0,2;0,0]);
>> [A2,b2,Aeq2,beq2]=vert2lcon([1,1;2,1;1,2]);
>> V = lcon2vert([A1;A2],[b1;b2],[],[])
Something's wrong. We should have found a recession vector (bb<0).

@Bhimavarapu,
The code doesn't generate any plots, but there are other utilities on the FEX that you could use for plotting polyhedra, e.g.,
<http://www.mathworks.com/matlabcentral/fileexchange/9261-plot-2d-3d-region>
As for "not getting the vertices", if you mean you are getting empty output V=[], it means your polyhedron appears empty to the code. Hard to say more without seeing what you're doing.

Cong,
Points on the boundary of the polytope are expected to violate the inequalities by small amounts due to finite precision arithmetic. When I compute the violations for your A,b data with the code below, I find that they are indeed very small O(1e-13),
>> V=lcon2vert(A,b);
>> slacks=bsxfun(@minus, b,A*V');
>> violations = slacks(slacks<0)
violations =
1.0e-13 *
-0.1066
-0.0355
-0.5684
-0.0711
-0.0355
-0.0355
-0.5684
-0.0355
-0.2842
-0.5684
-0.1066
-0.2487
-0.3375

Comment only